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ABSTRACT 



DUDLEY KNOX LIBRARY 
NAVAL POSTGRADUATE SCHOOL 
MONTEREY CA 93943-5101 



The Janus simulation model was initially designed to operate in a stand-alone 
mode. There is an ongoing research project to link Janus to other constructive 
simulations and virtual simulators. The present standard used to connect different models 
is Distributed Interactive Simulation (DIS). Janus can operate in a DIS environment 
using a cell adapter unit called the World Modeler. The combination of Janus and the 
World Modeler is known as JLink. A goal of the JLink system is to replicate the 
analytical and training fidelity of stand-alone Janus in a distributed exercise. The 
purpose of this thesis is to assess the current state of JLink development. 

The experiment simulated three scenarios: armored, armored coalition, and light 
infantry battalions attacking against a defending company. All scenarios were executed 
in two contrasting environments. The simulation included the recently developed JLink 
features Family of Scatterable Mine (FASCAM) and chemical artillery. 

The thesis used five Measures of Performance to base the assessment: 1) 
FASCAM kills, 2) Chemical Artillery Kills, 3) Detection Ranges, 4) Kill Ranges, and 5) 
Loss Exchange Ratio. The statistical tests used for analysis were the Analysis of 
Variance (ANOVA) test, two-sample r-test, and Wilcoxon test. 

The results of the analysis show that JLink requires adjustments to artillery 
delivery methods in order to correct chemical artillery discrepancies and detection range 
issues. In general, JLink accurately portrays coalition warfare and satisfactorily replicates 
armored and infantry scenarios in contrasting environments. 
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EXECUTIVE SUMMARY 



The Army relies heavily on computer simulation as a training an analytical tool. 
One simulation the Army uses is Janus, a high resolution combat simulation model 
design to operate in a stand-alone mode. There is an ongoing research effort to link Janus 
with other constructive simulations or virtual simulators. The present standard for linking 
different models is Distributed Interactive Simulation (DIS). Janus can operate in a DIS 
environment using a cell adapter unit called the World Modeler. The combination of 
Janus and the World Modeler is known as JLink. A goal of JLink is to replicate the 
training and analytic fidelity of stand-alone Janus in a distributed (JLink) exercise. The 
level of fidelity may be gauged by comparing the results of a scenario run in stand-alone 
Janus with the results from the same scenario run in JLink. The purpose of this thesis is 
to assess the current state of JLink development by measuring the amount of distortion 
between stand-alone Janus and JLink. 

The experiment tested specific areas of JLink which had yet been assessed. 
Previously, only rudimentary scenarios had been tested in a single environment. This 
thesis tested scenarios using different types of combat units in two different 
environments. The thesis also tested JLink’ s ability to portray coalition warfare. Finally, 
the thesis assessed recent software inputs to JLink which simulated specific combat 
functions, in particular Family of Scatterable Mines (FASCAM) and chemical artillery. 
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The experimental design used to assess these aspects of JLink included armored 
and light infantry units. The first scenario involved a Soviet-equipped armored battalion 
attacking against a defending US-equipped armored company. The second scenario was 
identical to the first except that the attacking battalion was divided into two separate 
forces, both attacking the armored company. This scenario was used to assess JLink’ s 
ability to replicate a coalition battle. The last scenario involved a Soviet-equipped light 
battalion attacking against a US-equipped light infantry company. All three scenarios 
were executed in two different environments. The two environments selected were Fort 
Hunter Liggett, California (HL) and Southwest Asia (SWA). These two environments 
offered contrasting terrain, the first being wooded with rolling hills and the second being 
flat and open. Using different types of units in contrasting terrain permitted assessment 
of JLink’s ability to portray varied scenarios. To test the specific combat functions of 
FASCAM and chemical kills, the defending company used these weapons in every 
scenario. 

The Measures of Performance (MOPs) selected for analysis were FASCAM kills, 
chemical kills, detection range, kill range, rounds fired, and Loss Exchange Ratio (LER). 
These MOPs allowed for analysis of the basic procedures of an engagement. First, one 
entity must detect another entity. Once detected, the entity may elect to shoot at the 
target. Analyzing detection range, kill range, and rounds fired offers a quantitative 
method of assessing these basic engagement procedures. The LER is a function of the 
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other MOPs and measures the overall outcome of the battle. The MOPs FASCAM kills 



and chemical kills were used to assess JLink’ s ability to simulate those functions. 

The statistical tests used to analyze the data were the Analysis of Variance 
(ANOVA), two sample r-test, and the nonparametric Wilcoxon test. First the data was 
analyzed using the ANOVA test whether overall averages for a specific MOP were 
statistically similar between the Janus and JLink runs taking into account all scenarios 
and environments. The ANOVA was also used to determine interactions between the 
scenarios, environments, and the mode (Janus or JLink) for each MOP. In the case where 
the ANOVA determined that the Janus and JLink means were statistically different, pair- 
wise two sample r-tests were used to identify where the differences in the means existed. 
Finally, the nonparametric Wilcoxon was also used in a pair-wise method to test those 
populations which did not satisfy r-test assumptions and to substantiate the r-test results. 

The results of the analysis show that JLink requires adjustments to artillery 
delivery methods in order to calibrate chemical artillery discrepancies and detection range 
issues. JLink chemical artillery killed far fewer entities than Janus, and JLink 
consistently detected farther than Janus. The finding is that to match Janus, JLink must 
simulate an artillery volley as one large cloud, like Janus, as opposed to individual 
rounds, as dictated by DIS standards. In general, JLink accurately portrays coalition 
warfare and satisfactorily replicates armored and infantry scenarios in contrasting 
environments. 
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I. INTRODUCTION 



The Army relies heavily on computer simulations for training and force analysis. 
One simulation in use today is Janus, an event-driven wargame named for the two-faced 
Roman god Janus, who was the guardian of the portals of Rome and the patron of 
beginnings and endings. For nearly three decades, Janus has proven to be a valuable 
training and analysis tool and has evolved into a legacy model with many validated 
features. The current task is to link Janus, which is designed to operate in a stand-alone 
mode, with other live, virtual, or constructive models. 

The present standard used to connect two or more models is known as Distributed 
Interactive Simulation (DIS). Janus is capable of operating in a DIS environment using a 
cell adapter unit called the World Modeler. The World Modeler is software, designed to 
run on a low-end Silicon Graphics computer, which performs functions required by the 
DIS architecture that are not performed by Janus. The combination of Janus and the 
World Modeler is known as JLink [Ref. l:p.l]. 

JLink combines the training and analytic potential of a widely fielded constructive 
simulation, Janus, with a DIS-compatible simulation. A goal of the JLink system is to 
replicate the analytical and training fidelity of a stand-alone Janus in a distributed (JLink) 
exercise. This level of fidelity may be gauged by comparing the results of a scenario run 
in stand-alone Janus with the results of the same scenario run in JLink, using similar 
starting conditions. The purpose of this thesis is to assess the current state of JLink 
development by measuring the amount of distortion between stand-alone Janus and 
distributed Janus using JLink, by applying statistical analysis. The assessment will aid in 
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identifying credible areas of JLink as well as aspects of JLink which require further 
development. As interest in JLink continues to grow throughout the Army and across 
services, improvements in JLink fidelity will prove valuable and return many training and 
analytical benefits. 

A. JANUS 

1. Janus Background 

Lawrence Livermore National Laboratory originally developed Janus in 
the 1970s, with the current version developed by Training and Doctrine Command 
Analysis Center (TRAC) at White Sands Missile Range, New Mexico. Janus is a 
stochastic, interactive, high resolution, multisided, model with a robust database and a 
detailed post-processor for gathering results. 

Being a stochastic model, Janus uses a random number generator and 
probabilities to determine the outcome of detections and engagements. Janus random 
number seeds range from 1 to 99. Prior to executing a scenario, the user may set the 
random number seed manually or elect to have Janus randomly select the seed. Since 
Janus is an event based model, the user can achieve exact replication if the same random 
number seed is selected and there is no human interaction. 

The interactive nature of Janus permits its users to make decisions during 
the scenario to influence the outcome of the battle. Due to the variability of user inputs 
between scenarios, interaction will produce different results for two Janus scenarios using 
an identical random number seed. Examples of interactive functions include creating or 
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altering movement routes, planning and firing artillery missions, and 
mounting/dismounting units. User interaction is not necessary for pre-planned scenarios 
designed to fight two opposing forces. For purposes of this thesis, all scenarios were pre- 
planned and included no interaction. 

Janus is a high resolution model because it represents entities down to 
individual systems. However, when practical the user may aggregate entities, with 
artillery systems typically being aggregated up to 40 entities [Ref. 2:p.21]. The situation 
and scenario will generally dictate the level of aggregation, with Janus users often 
aggregating to the lowest useful level in order to take advantage of Janus’ high-resolution 
capabilities. 

Janus can also model up to six different sides for a given scenario, to 
include simulating fratricide. Janus offers the option to separate the sides among 
different workstations (primarily for training) or play all sides on one workstation 
(primarily for analysis). 

Janus possesses a robust database which permits the user to define a weapon 
system extensively or capture the detailed factors required for the scenario. The database 
is divided into six main sections: systems, weapons, sensors, engineer data, weather data, 
and chemical and heat data. Each main section has numerous subsections, all requiring 
inspection prior to creating a scenario to ensure accurate information is provided. A 
separate Janus database Manager’s Manual is available and necessary for reference. 

Janus’ post-processor permits the analyst to gather detailed results from 
scenarios. The post-processor offers a variety of detailed statistics, including reports on 
the number of rounds fired, kill ranges, detection ranges, minefield crossings, chemical 
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casualties, and Loss Exchange Ratios (LER). The user has the option to select which 
reports will appear in the post-processor, which precludes generating unnecessary reports 
while focusing on the area of interest. The reports of interest for this thesis are those 
reports mentioned above. 

2. Janus Uses 

Originally designed as an analytical model, Janus is presently used in the 
Training, Exercises, and Military Operations (TEMO) and the Advanced Concepts 
Requirements (ACR) domains. Both domains participate as members of the 
configuration control board to ensure all of the users’ requirements are satisfied. 

a. Training 

Janus is best geared to train leaders at brigade level and below 
because of its high resolution capabilities. The ability to model down to an individual 
system enables scenarios to focus on company, platoon, or squad size elements. 
Commanders at brigade level can use Janus to train the leaders of these elements in the 
decision-making process. Janus’ interactive capabilities allow leaders to make decisions 
during an engagement, and then to observe the outcome of their actions. Following the 
scenario, the entire battle can be replayed using the Janus Analyst Workstation (JAWS) 
and assessed for after-action review purposes. 
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b. Analysis 



Janus is particularly suited for the analyst. Janus’ post-processor 
permits the analyst to gather results from scenarios for further study of topics such as new 
tactics, techniques, and procedures. The analyst creates scenarios using standard 
methods, and then compares the results to a scenario using newly proposed methods. 

The analyst also uses Janus to study the effects of a new or modified 
weapon system. Janus’ database is robust enough to capture or alter many aspects of a 
weapon system, including weapon range, detection range, sensor type, travel speeds, and 
crew size. These weapon systems can also be tested in various environments and weather 
conditions based on the weapon’s mission profile. 

As a stand-alone simulation, Janus has aided Army trainers and analyst for 
years. Janus must now move to the next level of simulation, the DIS environment, in 
order to remain beneficial to the Army of the 21 st century. 

B. DISTRIBUTED INTERACTIVE SIMULATION (DIS) 

1. DIS Background 

The DIS Master Plan defines DIS as “A synthetic environment 
within which humans may interact through simulation(s) and/or simulators at multiple 
networked sites using compliant architecture, modeling, protocols, standards, and 
databases” [Ref. 3:p. I- 1 ]. Leading edge computer technologies and advanced 
communications networking allow for such an environment to exist. Essentially, DIS is 
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an environment which brings the power of the computer and communications together to 
benefit the trainer and analyst. 

The terms of DIS are defined as follows: Distributed refers to geographically 
separated simulations, each hosted on a computer and connected via networks to create a 
shared environment. Interactive is described as different simulations electronically linked 
to act together and upon one another. Simulation is divided into three categories: a) live 
simulation where actual equipment and soldiers are operating in the field, b) constructive 
simulation which includes wargames and models such as Janus, and c) virtual simulation 
which involves manned simulators interacting within a virtual reality environment [Ref. 
3:p. 1-2 - 1-3]. 

The primary objective of the DIS program is to establish an architecture which 
permits the linkage of different simulation environments into a seamless synthetic world. 
The infrastructure brings together systems built for separate purposes, technologies from 
different eras, products from different vendors, and platforms from various services [Ref. 
3:p. 1-2]. Janus, operating in a DIS environment at Ft. Hood, Texas, linked to a virtual 
AH-64 Apache simulator at Ft. Hood and a constructive ModSAF simulation at Ft. 

Knox, Kentucky is a realistic application of the DIS objective (described above). 

2. DIS Benefits 

Since the development of DIS in 1991 with the successful linking and 
interacting of the Simulation Network (SIMNET) program, observers have 
enthusiastically embraced the benefits of DIS. Specifically, a DIS environment can 
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benefit the Army and other services in the areas of training, combat development, and 
systems acquisition. 

In terms of training, commanders have greater potential to train as they fight. If a 
DIS capability is established between live and constructive and/or virtual simulations, 
training will more realistically mirror combat conditions. Another aspect of DIS training 
benefits is in the area of jointness. An objective of DIS is to link simulations across 
services. Such a link enhances joint training and promotes increased joint training 
activities. 

DIS provides the combat development community expanded capabilities. First, 
DIS provides the ability to introduce new simulations into the synthetic DIS environment, 
such as an individual soldier simulated as a virtual entity, with minimal impact on 
software. DIS also provides combat developers the capability to collect and record 
actions and reactions generated by humans with minimal interference from observers 
[Ref. 3:p. HI-4]. 

The benefit DIS brings to systems acquisition is reduced test and evaluation costs. 
By linking live, constructive, and virtual simulations, the acquisition life cycle can be 
reduced significantly, resulting in test and evaluation cost reductions. 

The focus of this thesis is evaluating Janus in a DIS environment. The value of 
Janus and the importance of DIS were the driving forces in the development of JLink, the 
method by which Janus operates in the DIS environment. The next section highlights the 
methods of JLink and the challenges faced by JLink. 
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C. JLINK 



1. JLink Background 

In 1993, TRAC Monterey began developing the JLink system to support the Anti- 
Armor Advanced Technology Demonstration (A2ATD). The A2ATD was charged with 
developing and demonstrating the use of DIS to evaluate anti-armor weapon systems on a 
combined arms task force. TRAC Monterey had the overall responsibility for building 
the systems, providing project management, and contributing functional area expertise. 
Other contributors to the JLink project include the Naval Postgraduate School (NPS), 
RAND, MITRE, and Rolands and Associates (R&A). NPS and RAND developed the 
interface between Janus and the World Modeler. RAND also conducted terrain 
conversions and modified Janus internal algorithms as necessary, while MITRE focused 
on implementing dead reckoning algorithms. [Ref. l:p. 1] 

Following the A2ATD, TRAC Monterey continued to develop JLink to serve the 
TEMO and ACR domains with future training and analytical requirements. To assess the 
state of JLink development, TRAC Monterey conducted a comparison study of Janus to 
JLink in the summer of 1996. The study applied a rudimentary scenario in a single 
environment to test basic engagement methodologies. The results of the study showed 
that JLink produced similar kill ranges and loss exchange ratios when compared to Janus, 
but statically different shot totals for the blue forces. The cause of the total shots 
discrepancy was identified as a z-coordinate issue with respect to aircraft, and the study 
concluded that a scenario fought in stand alone Janus is “closely matched” with the same 
scenario fought under JLink’s current configuration [Ref. 4:p. 9-10]. 
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As TRAC continues to develop JLink, recurrent testing is necessary to assess 
JLink’s current state. As JLink evolves into a robust model, capable of simulating 
multiple battlefield phenomena, comparison studies between Janus and JLink must be 
diverse enough to test and analyze newly developed aspects of the system. This thesis 
conducts such a test by simulating large scale scenarios in multiple environments. The 
experiment also includes recently developed JLink functions such as Family of 
Scatterable Mines (FASCAM) and chemical artillery. 

2, JLink Architecture 

This section describes the JLink architecture, which enables Janus to 
operate in a DIS environment through use of the World Modeler. Network management 
establishes the connection between Janus and other DIS-compatible models, such as 
ModSAF, using two different protocols. The Transmission Control Protocol/Intemet 
Protocol (TCP/IP) is used to establish communications between the World Modeler and 
Janus, while the User Datagram Protocol (UDP/IP) is used to establish communications 
between the World Modeler and DIS. Through these links, the World Modeler controls 
dataflow by passing Protocol Data Unit (PDU) information to Janus and DIS at the 
appropriate time intervals [Ref. l:p. 5] (Figure 1). The purpose of the PDU is to 
facilitate the electronic transfer of data between simulations with different software. 
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Figure 1. JLink Design 



Suppose Janus is portraying Blue forces and a DIS-compatible simulation is 
portraying Red forces. When Blue generates an event, such as firing at a Red entity, 
Janus sends a message to the World Modeler via the TCP/IP connection. The World 
Modeler uses “hooks” to capture the Janus protocol and converts the Janus protocol to a 
DIS PDU, which can be processed by a DIS model. The World Modeler then broadcasts 
the DIS PDU via the UDP/IP connection to the network for use by the DIS (Red) 
simulation. Conversely, when the Red force generates an event, the corresponding DIS 
PDU is sent via the network to the World Modeler, where it converts the DIS PDU to a 
Janus protocol, which in turn is sent to the Janus simulation for processing. In essence. 
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one of the functions of the World Modeler is to convert Janus Protocols to DIS PDUs and 



vice versa. 

Currently, JLink has successfully interacted with several constructive simulations 
and virtual simulators, via DIS PDUs. However, evolving systems require extensive 
testing, and the JLink process frequently requires adjustments or model calibration to 
achieve effective levels of interoperability between interfacing systems. 
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II. PROBLEM DESCRIPTION 



Improved interoperability can be translated into minimizing the distortion between 
results from Janus stand-alone and the results of the same scenario run in JLink. Two or 
more simulations/simulators are defined as “DIS interoperable” when their performance 
characteristics support a fair fight to the fidelity required for the exercise. A fair fight 
exists when the differences in two simulations’ performance are overwhelmed by user 
actions [Ref. 5:p. A-4], There are two general factors which may contribute to 
differences in the results between Janus stand-alone and JLink scenarios and lead to an 
unfair fight. These factors are the JLink architecture and the DIS PDU itself, which form 
the basis for the problem description of this thesis. This chapter addresses how JLink 
architecture and the DIS PDU may contribute to distortion. As part of the problem 
description, the chapter concludes by identifying specific features of JLink, which may be 
affected by the factors mentioned above, that require testing. 

A. DISTORTION FROM JLINK ARCHITECTURE 

Inherent aspects of the JLink architecture may contribute to differences in 
JLink results versus Janus stand-alone results. Given that Janus is an event-driven model, 
the sequence of events is vital to the outcome. In stand-alone Janus, a scenario will 
produce the same sequence of events, and hence the same results, for every run as long as 
the random number seed remains unchanged and no human interaction occurs. In JLink, 
delays in passing and converting PDUs may produce a different sequence of events than 
the corresponding Janus stand-alone scenario. Delays in passing PDUs between DIS 
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models are a function of network congestion and quality of the network hardware. 

Delays in converting Janus protocols to DIS PDUs, and vice versa, also contribute to an 
altered sequence of events. A different sequence of events will result in different 
outcomes. Say for instance a Blue entity in stand-alone Janus generates a shot event on 
the event list against a Red entity at time =10 minutes, resulting in a Red kill at time = 1 1 
minutes. In JLink, the same shot event may not get scheduled until time =12 minutes 
due to delays in passing and converting the appropriate PDU, which may result in the Red 
entity surviving (and shooting) longer than in Janus stand-alone. 

Another aspect of JLink architecture contributing to distortion is the issue of 
random number generators. Janus stand-alone utilizes only one random number 
generator to determine the outcome of all stochastic events. JLink uses the Janus random 
number generator to adjudicate only those events corresponding to its own side. The 
other sides represented in the scenario use their own random number generators to 
adjudicate stochastic events. Even if all random number generators involved behave 
identically (which is unlikely) and they are all initiated with the same random number 
seed, they will not produce the same aggregate random number string as the Janus stand- 
alone generator for an entire scenario. Dissimilar random number strings will result in • 
dissimilar results between Janus stand-alone and JLink. 

This thesis will not attempt to determine whether any JLink architecture issues 
like those above contributed to distorted results. The thesis experiment did, however, 
reduce the impact of the architecture on results by using the following procedures. To 
reduce interference when passing PDUs, all JLink runs were executed during times when 
the TRAC Monterey network was free from all other activity. Also, all scenarios run in 
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JLink used the same random number seeds used in Janus stand-alone. Using the same 
random number seeds will produce some of the same random numbers for both JLink and 
Janus stand-alone scenarios. 

B. DISTORTION FROM DIS PDUs 

One major challenge to World Modeler developers is to accurately translate and 
create DIS PDUs as they flow to and from the Janus model. The DIS PDUs created by 
the World Modeler for use by other distributed systems must portray the originating Janus 
protocol as accurately as possible. Likewise, DIS PDUs sent to the World Modeler by 
distributed simulations must be translated appropriately for use by Janus. Inaccurate or 
inappropriate conversions of PDUs may produce distorted JLink results as compared to 
the Janus stand-alone standard. 

One example where appropriate conversions of PDUs is critical is the flight 
altitude of rotary-wing aircraft. Janus only portrays two flight levels, treetop level (low) 
and at altitude (high), whereas ModSAF simulates continuous altitudes. The issue 
becomes at which flight level a ModSAF helicopter entity should be simulated in Janus, 
and at what altitude a low flying Janus helicopter should be simulated in ModSAF. 

In order to test for distortion caused by the DIS PDUs, JLink requires additional 
testing of a wide variety of features. To date, JLink fidelity has only been tested using 
rudimentary scenarios in a single environment. JLink development is now at the stage 
which can support testing of additional aspects of the system. 
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C. JLINK FEATURES TESTED IN EXPERIMENT 



1. Varied Scenarios 

One aspect of the JLink system this thesis tests is JLink’s ability to 
accurately portray different types of combat units in the offense or defense, negotiating 
obstacles, under different environmental conditions. Such scenarios have yet to be 
validated using JLink. If JLink operates as designed, the results from these varied 
scenarios will be similar to the results of the same scenarios run in stand-alone Janus. 

2. Coalition Warfare 

The thesis experiment also includes an assessment of JLink simulating the 
synergistic effects of coalition warfare. Previous tests involving JLink have only 
included two sides fighting against one another. As part of the experiment, one scenario 
includes three sides, two forces allied together fighting against the third. Pitting two sides 
against a lone third side may generate insightful results regarding JLink’s ability to 
accurately fight a coalition battle. 

3. Specific Combat Functions 

In addition, the thesis tests whether JLink accurately portrays the effects of 
specific combat functions. Recent JLink software inputs, designed to replicate specific 
combat functions such as chemical artillery delivery methods and FASCAM, require 
testing. In stand-alone Janus, chemical artillery volleys are delivered as an aggregated 
threat, with one large radius. DIS standards require that individual artillery rounds be 
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passed between simulations. Because of this requirement, programmers have modeled a 
JLink chemical artillery volley as having each round create its own radius of effect. 
Figure 2 depicts a volley of three artillery rounds and illustrates the difference in the two 
delivery methods. 




Figure 2. Artillery Delivery Methods 



The thesis will determine whether such a distribution of chemical artillery rounds 
significantly affects the number of kills. 

Another specific combat function requiring testing is FASCAM, an 
artillery-delivered minefield. Unlike chemical artillery, aggregation is not an issue 
regarding FASCAM. This thesis provides initial feedback on the FASCAM code as to 
whether it generates the same number of kills as FASCAM in stand-alone Janus. 

Herein lies the goal of this thesis — to assess JLink fidelity by testing whether 
JLink produces similar results to stand-alone Janus using varied scenarios and coalition 
warfare, and by determining whether recent software inputs to specific combat functions 
are providing accurate representations. 
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III. DESIGN OF EXPERIMENT 



The overall goal of the experiment was to provide a means to obtain data from a 
scenario executed in Janus and the same scenario executed in JLink for purposes of 
analysis. The issues involved in meeting the goal were selecting appropriate data to be 
analyzed, determining the structure of the scenarios, and conducting the experiment. 

This chapter begins by presenting the measures of performance and the methodology 
supporting their selection. Next, details of the scenarios are described, followed by a 
discussion of the conduct of the experiment and issues encountered with the execution. 
The experiment included necessary conditions for testing the issues defined in Chapter II. 

A. MEASURES OF PERFORMANCE 

The first issue in designing the experiment was to identify what data would be 
analyzed. The MOPs were limited only by the information provided in the post- 
processor. For clarity, this thesis refers to the data as measures of performance, as 
opposed to measures of effectiveness. The reason for the distinction is that the purpose of 
the thesis is not to analyze the effectiveness of a new weapon system or perhaps a tactic. 
Rather, the focus is to assess JLink fidelity by comparing the performance of the JLink 
system to the performance of the stand-alone Janus system. 

The MOPs were derived from issues raised in the Problem Description. The areas 
that require testing in JLink are varied scenarios, meaning scenarios using different types 
of units in contrasting environments, coalition warfare, and specific combat functions 
such as chemical artillery and FASCAM. The determination whether JLink accurately 
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portrays the areas defined in the Problem Description is derived from the analysis of one 
or more of the following MOPs: 1) Detection Ranges, 2) Kill Ranges, 3) Rounds Fired, 
4) Loss Exchange Ratio (LER), 5) Chemical Artillery Kills, and 6) FASCAM Kills. 

1. MOPs for Varied Scenarios and Coalition Warfare 

To determine the fidelity of varied scenarios in JLink, the thesis analyzed 
detection ranges, kill ranges, rounds fired, and LER. Applying appropriate MOPs to the 
broad topic of scenario fidelity is critical to arriving at valid conclusions. The analysis 
must focus on the basic procedures of an engagement. First, one entity must detect 
another entity. Comparing the detection ranges of Janus stand-alone and JLink is a 
quantitative method of assessing the detection process in JLink. Next, the detecting entity 
may elect to shoot at the target, possibly resulting in a kill. Analyzing the kill ranges 
between Janus and JLink is a quantitative method for assessing the killing process in 
JLink. Since not all shots result in kills, the total number of rounds fired must also be 
analyzed. The post-processor also provides the flexibility to analyze detection ranges, kill 
ranges, and rounds fired for an entire scenario or by individual side. For instance, rounds 
fired can be analyzed by comparing total Janus rounds fired to total JLink rounds fired for 
the same scenario, or by comparing blue Janus rounds versus blue JLink rounds. 

Detecting and killing the target are the basic elements of a battle. If 
JLink replicates these functions in the likeness of Janus, one would expect similar overall 
outcomes between Janus stand-alone and JLink scenarios. Further, since LERs are 
primarily a function of detections and kills, the LERs offer a quantitative method to 
assess the overall outcome of a JLink scenario. 
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2 . 



MOPs for Chemical Artillery and FASCAM 



To determine whether JLink achieves acceptable results from chemical 
artillery and FASCAM, the thesis analyzed the number of kills generated by chemical 
artillery and the number of kills resulting from firing FASCAM. 

B. SCENARIOS 

The scenarios were determined based on the Problem Description requirements. 
Specifically, the thesis must analyze results from varied scenarios, coalition warfare, and 
the specific combat functions of chemical artillery and FASCAM. US and Russian-style 
armored and light infantry units, along with their supporting elements, were a logical 
selection for the unit types. Armored and light infantry units are combat arms likely to be 
involved in offensive or defensive operations. Units equipped with US- and Russian- 
made weapons bring diverse capabilities to the battlefield, adding robustness to the study. 

In determining the environments, the goal was to select two contrasting locations 
which will likely have different effects on detection ranges and engagement ranges. The 
two contrasting environments used were Southwest Asia (SWA) and Hunter Liggett, 
California (HL). SWA is primarily desert with no trees and slightly undulating terrain, 
which provides some relief. HL is comprised of small hills and lightly wooded terrain. 
The relatively flat terrain of SWA compared to the rolling, wooded terrain of HL offers 
contrasting settings to test sensor functions and weapons platforms. 

After determining the types of units and environments, the final element in the 
scenario structure was the means of engagement. The engagement used was an attacking 
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battalion against a defending company. Such an engagement provided results from units 
with contrasting postures, moving as opposed to stationary. Different postures affect the 
detection and firing capabilities of entities in Janus. 

Once all of the elements of the scenarios were identified, three scenarios were 
developed. The first scenario involved a US-equipped armored company (blue) 
defending against an attacking Russian-equipped armored battalion (red). The second 
scenario was identical to the first, except that the red force was divided into two sides to 
form a coalition. The red coalition attacked the (defending) blue force to test the results 
of a coalition engagement, in both a stand-alone and distributed mode. The third scenario 
involved a US-equipped light infantry company (blue) defending against an attacking 
Russian-equipped light infantry battalion (red). In every scenario, the blue side fired 
chemical artillery rounds and FASCAM at specifically identified red units traveling on a 
narrow avenue of approach. This was done to isolate the weapons’ performance to aid in 
the deconfliction of MOPs for analysis. Also, all blue sides possessed obstacles to aid in 
the defense. The obstacles included minefields and abatis for road barriers. 

1. Scenario 1 

The weapon systems used by the blue side in scenario 1 were the MIA 1 
tank, M2 Infantry Fighting Vehicle (IFV), Multiple Launcher Rocket System (MLRS) 
artillery, AH64 Apache helicopter, and four riflemen as scouts. The red side consisted of 
the T72 tank, BMP-2 IFV, MLR 16 (MLRS equivalent). Hind helicopter, and SAM air 
defense weapon. The blue side consisted of 32 total units (mainly MlAls), while the red 
side had 91 total units (mainly T72s), resulting in approximately a 3: 1 ratio in favor of the 
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attacker. The scenario was designed to emphasize the capabilities of the M1A1 and T72 
tanks, with support from the other systems. 

2. Scenario 2 

The weapon systems, weapon numbers, and scenario design in scenario 2 
were identical to those in scenario 1. In scenario 2, the red force was divided into two 
sides, fighting together as a coalition against the blue force. 

3. Scenario 3 

The weapon systems used by the blue side in scenario 3 were the rifleman with 
M16 rifle, rifleman with machine gun, Light Anti-tank Weapon (LAW), mortars, and 
MLRS. The red side consisted of the rifleman with AK47 rifle, rifleman with machine 
gun, anti-tank RPG, mortars, and MLR 16. The blue side had 116 units versus 653 units 
for the red side, resulting in a 5.6: 1 edge in favor of the attacker. The scenario 
emphasized the capabilities of the rifle and machine gun. 

4. Scenario General Characteristics 

Red forces attacked from north to south in all scenarios since likely avenues of 
approach in both environments favored north-south attack routes. Recall that blue forces 
fired chemical rounds at specific red units to assess their effectiveness in distributed 
simulations. Since the wind generally travels from southeast to northwest in both 
environments, it was logical to have the blue side firing northward, or downwind. Hence, 
the red forces were arrayed to attack south. 
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Another common characteristic between scenarios was the placement of the blue 
forces. All blue units were placed on relatively defensible terrain, oriented to the north. 
Ensuring the blue units had adequate Line of Sight (LOS) to the north was critical to the 
defense. Figure 3 shows the Janus map of HL, with the blue units in position. The figure 
also shows the LOS of one blue entity, with “blind spots” shown as broken radials. 




Figure 3. Scenario Graphics 



Additionally, the scenarios were designed so that all blue entities could fire at all 
red entities, and vice versa. This design required slight adjustments to the database in 
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some instances. The purpose of this design was to test JLink’s ability to portray multiple 
types of weapons engaging each other. 

Finally, all scenarios lasted between 17-27 minutes, with the average length being 
approximately 20 minutes. Although the test scenario is short compared to typical Janus 
exercises, which often last several days, 20 minutes permitted artillery units to fire 
assigned volleys, “pucked” FASCAM to fire five minefields, aircraft to fly designated 
routes, and attacking elements to move into and through the engagement area. 

C. EXPERIMENT 

1. Conduct of Experiment 

Each of the three scenarios were executed in two environments, SWA and HL, 
resulting in six combinations. Each combination was then run in two different modes. 
First they were run in the Janus stand-alone mode, Janus (blue) versus Janus (red), with 
both the blue and red forces simulated on the same Hewlett-Packard (HP) workstation. 
Then the same six combinations were run in the JLink (blue) versus JLink (red) mode, 
with the two forces simulated on separate workstations, as illustrated in Figure 4. The 
JLink versus JLink mode is the method developed at TRAC Monterey to test distributed 
Janus without having to connect to another type of DIS simulation. Each Janus terminal 
is connected to a World Modeler which in turn is connected to the network. 
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JUNK 



Figure 4. JLink vs JLink 



Six combinations run in two different modes resulted in 12 total combinations in 
order to capture all scenarios, environments, and modes. Figure 5 depicts the overall 
experiment. 





Janus Stand-Alone 


JLink vs JLink 


Armored SWA 
Armored HL 


(avg of 10 runs) 
(avg of 10 runs) 


(avg of 10 runs) 
(avg of 10 runs) 


Armored Coalition SWA 
Armored Coalition HL 


(avg of 10 runs) 
(avg of 10 runs) 


(avg of 10 runs) 
(avg of 10 runs) 


Light Infantry SWA 
Light Infantry HL 


(avg of 10 runs) 
(avg of 10 runs) 


(avg of 10 runs) 
(avg of 10 runs) 



Figure 5. Overall Experiment (Averages for Specific MOP) 



These 12 combinations, however, only provide one data point for each 
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MOP, and will be referred to as a single run of the experiment. Ultimately, 10 runs were 
performed for each cell to provide a reasonable sample. (The issue regarding the total 
number of runs is addressed in Chapter III, paragraph C. 2.) 

The desired method of analysis was to compare the MOPs resulting from a 
specified number of Janus stand-alone runs to the MOPs resulting from the same number 
of JLink runs. Since a stand-alone run will produce the exact same results if the same 
random number seed is sown, all stand-alone runs were executed using different, 
randomly selected, seeds. The same random number seeds were then used in the 
corresponding JLink runs. Also, due to the inherent variability caused by JLink 
architecture, 10 additional runs were performed in JLink using the same random number 
seed for all 10 runs to isolate the variance contribution from that aspect of the JLink 
system. 



2. Experiment Issues 

As mentioned previously, all JLink runs were done during times of minimal traffic 
on the TRAC Monterey network. In all but four JLink runs, the scenarios were executed 
when the network was completely free of other users. Executing the scenarios with 
minimal network interference reduces variability in the results due to event scheduling. If 
the network is congested, delays may occur in passing information and the sequence of 
events may be affected. 

Determining the sample size is fundamental to the design of every experiment. 

An experiment is conducted a number of times so that the data produces good estimators 
of the true population parameters. Different techniques are available to arrive at a 
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satisfactory sample size. The thesis proceeded as follows regarding this issue. First, a 
practical approach was used. Based on experience and recommendations from senior 
Janus analysts, a sample size of n = 10 generally produces results with “acceptable 
variance” for the MOPs defined in this thesis. That is, the experiment is run until an 
estimate of the variance for the mean is reduced to a pre-determined, acceptable level. 

This level will be different for each measure of performance. Based on the raw data 
figures in Appendix A, acceptable variance was achieved in detection ranges, kill ranges, 
LERs, and for most cases of rounds fired. Discussion in Chapter IV highlights anomalies 
in FASCAM and chemical kills which preclude the possibility of reaching acceptable 
variances levels for these MOPs with reasonable sample sizes. 

The thesis also considered the issue of normality of the data when 
determining sample size. Normality of the data set is one assumption in the two sample /- 
test, which is used for analysis of the MOPs. The normal distribution quantile plots in 
Appendix F show that the results for most MOPs are relatively normal, with some 
obvious exceptions. Also, the /-test normality assumption is generally satisfied if the 
sample averages are normally distributed. As the sample size increases, one would 
expect the sample averages to become more normal, based on the Central Limit Theorem 
[Ref. 6:p. 232], In this case, we expect a sample size of n=10 to be large enough to 
produce normally distributed averages. 

Another issue addressed prior to beginning the runs was that the user can only fire 
FASCAM while the scenario is on-going. The users timing and accuracy could vary 
significantly from run to run and thus alter the FASCAM results. The solution was 
discovered in the “puck” run. In essence, a puck run records all of the actions of a 
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previously run scenario, including the user actions, and replays them on the current 
scenario run. By pucking the initial run of each scenario, then changing the random 
number seeds for the following runs, the user can replicate the same scenario, to include 
FASCAM volleys, while generating different results due to the new seed. 

The final issue encountered in conducting the experiment occurred after the runs 
were complete. The issue involved gathering the information from the post-processor 
into a usable form. After transferring the post-processor information to a personal 
computer, extensive manipulation of the data was required prior to importing the data 
into a spreadsheet or statistical package. Since the results from the post-processor are 
lengthy and involve thousands of numbers in the case of these scenarios, manipulating the 
data was cumbersome and occasionally resulted in lost data. To resolve the issue, a C 
program was written and used as a filter to easily gather the required data for the MOPs. 
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IV. DATA ANALYSIS 



This chapter presents the methodology for analyzing the results, the statistical 
tests used in the analysis, and the results from the analysis. The following chapter 
provides a detailed analysis of the results presented in this chapter. 

A. ANALYSIS METHODOLOGY 

After running the 1 2 combinations described in Figure 5, with 10 replications for 
each cell, summary statistics for each MOP were gathered. The raw data for the 
experiment is presented in Appendices A, B, and C. The LER is the total red entities 
killed divided by the total blue entities killed. Each data entry for detection range and kill 
range is the average of all detection ranges (kill ranges) for the specified mode on the 
given run. The rounds fired data is the total of all direct fire rounds shot for the specified 
mode on the given run. 

The data was then analyzed by MOP. First, the Analysis of Variance (ANOVA) 
test was used to determine if differences existed between the means of the Janus 
population and the JLink population for the given MOP. ANOVA determines whether 
sample means from different populations are statistically similar by analyzing the ratio of 
the variance estimates. ANOVA also helps identify interactions occurring between the 
three factors of the experiment - scenario type, environment, and mode (Janus/JLink) - 
with respect to the given MOP. 

The ANOVA test identifies whether their are differences in the means between 
the two populations, but does not indicate where the difference occurred. To identify 
where the differences in the means exist, a pair-wise two sample /-test was applied to 
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those MOPs whose populations were deemed statistically different by the ANOVA. In 
this case, the goal was to determine whether the results, from an MOP in a specific 
scenario and environment executed in Janus stand-alone, are statistically similar to the 
results of the same MOP under the same conditions executed in JLink. Such a pair-wise 
comparison highlights where differences in the means exist. 

For this thesis, the sample size is relatively small and the population variances are 
unknown, hence a two sample r-test is appropriate in this case. [Ref. 6:p. 357-359]. The 
assumptions for the two sample r-test are: 

1 . Both samples are independent random samples from normal populations. 

2. The two populations have equal variance. 

Data sets were analyzed to determine whether assumptions were satisfied prior to 
applying the r-test. 

In cases where the assumptions were not satisfied, the data was analyzed in the 
same pair-wise method using the less powerful Mann-Whitney-Wilcoxon (Wilcoxon) 
nonparametric test. The Wilcoxon test does not require that the data satisfy the normality 
assumption above. The Wilcoxon test was also performed on the MOPs which did satisfy 
the assumptions to further substantiate the results of the t-test. 

The data analysis concludes with a discussion of the variability within the JLink 
system to determine if the main contributor to variance comes from within the JLink 
system or from within the simulation runs. 
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B. TOOLS FOR ANALYSIS 



1. ANOVA 

The ANOVA determines whether sample means from different populations are 
statistically similar and identifies interaction between factors. The design of the 
experiment resulted in the data being organized into a 3 x 2 x 2 fashion, meaning the 
experiment tested three factors, the first involving three levels, and the second and third 
factors involving two levels (Figure 5). The first factor, scenario type, was tested at three 
levels: armored, armored coalition, and light infantry. The second factor, environment, 
was tested at two levels: SWA and HL. The third factor, mode, was tested at two levels: 



Janus stand-alone and JLink. The first and second factors were blocking factors, while 



the third factor acted as the treatment factor which this thesis intended to analyze. 



An interpretation of the ANOVA table follows. Table 1 shows the results of the 



ANOVA for detection ranges by the blue forces. 



Analysis of Variance Table for Detection Range Blue 
Response: MOE (Detection Range Blue) 

Terms added sequentially (first to last) 





Df 


Sum of Sq 


Mean Sq 


F Value 


Pr (F) 


Seen 


2 


95.34164 


47.67082 


6050.279 


0 . 0000000 


Env 


1 


2.51797 


2.51797 


319.576 


0.0000000 


Mode 


1 


4.11438 


4.11438 


522.188 


0,0000000 


Seen : Env 


2 


0 . 19275 


0.09638 


12.232 


0.0000163 


Seen .-Mode 


2 


1.64627 


0.82313 


104.470 


0 . 0000000 


Env : Mode 


1 


0.09369 


0.09369 


11.892 


0 . 0008050 


Seen : Env: Mode 


2 


0.00861 


0.00431 


0.547 


0.5805110 


Residuals 


108 


0.85094 


0.00788 







Table 1. ANOVA Results for Blue Detection Ranges 



The ANOVA table provides results for analysis of blocking factors, scenario type and 



environment, and the treatment factor, mode (Janus/JLink). The focus of the thesis is on 



the treatment factor, mode, to assess whether Janus and JLink produce similar results for 
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given MOPs. The highlighted row, labeled Mode, is interpreted as follows. The 
ANOVA model equation for Y ijk , the observation recorded for the z'th scenario, and y'th 

environment, and kth mode, is 

Y nk = V + A + r ] + Yk + e ikj 

where fl is the overall mean 

A is the effect of the z'th treatment 

tj is the effect of the y'th environment 

y k is the effect of the fcth mode 

£ jjk is the experimental error ~ Normal (0,cr 2 ), 

and it is initially assumed that there is no interaction between factors [Ref. 7: p. 532], In 
words, the recorded observations for Blue Detection Ranges, separated by mode, 
(highlighted row above) are interpreted as follows: 

Janus Blue Det Rng = overall mean Blue Det Rng + ScenEffect + EnvEffect + JanusModeEffect + error. 
Similarly, every JLink recorded observation is interpreted as follows: 

JLink Blue Det Rng = overall mean Blue Det Rng + ScenEffect + EnvEffect + JLinkModeEffect + error. 
Assuming the model is additive, subtracting the JLink equation from the Janus equation 

returns a model which isolates the MOP in terms of the mode effects (and error): 

Janus MOP - JLink MOP = Janus Mode Effect - JLink Mode Effect + error. 

In this case, the zero p-value in the Mode row means that the null hypothesis 

H a : mean Janus Blue Det Rng = mean JLink Blue Det Rng 
is strongly rejected. The ANOVA does not, however, identify where the difference 
occurred. 
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Figure 6 shows interaction plots for scenario:environment, scenario: mode, and 
environment:mode for the same MOP, blue detection ranges. The interpretation of the 
scen:mode interaction is as follows. The null hypothesis tested is 



H„ 



AvgJanusBlueDetRng — AvgJLinkBlueDetRng = A,scen#l 
AvgJanusBlueDetRng - AvgJLinkBlueDetRng = A, seen#! 
AvgJanusBlueDetRng — AvgJLinkBlueDetRng = A, seen#?) 



> AllA s the same. 



In this case, the p-value for scen:mode interaction is zero, thus rejecting the null 
hypothesis. This result can be interpreted as saying, not only are the sample means for 
blue detection ranges different between Janus and JLink, the difference varies between 
scenario. Figure 6 highlights with arrows where the difference in means are considerably 
larger between Janus and JLink for scenario #1 compared to the difference in scenario #3. 




Figure 6. Interaction Plots for Blue Detection Ranges 

The value of the interaction plots for this thesis is that they provide insights as to 
the cause of disparate results between Janus and JLink. For instance, the scen.mode 



35 




interaction plot shows that greater differences in blue detection ranges occur in armored 



and armored coalition scenarios as compared to light infantry scenarios. This finding 
indicates that situations peculiar to armor and armor coalition scenarios contribute most 
to the gap in blue detection ranges between Janus and JLink. 

2. Two Sample /-Test 

a. Test Description 

The two sample /-test tests the following hypothesis: 



In this case, the thesis determines whether the two sample averages are statistically 
similar, so our A = 0. The test statistic used is 



where s p is the pooled sample variance, defined at the beginning of this chapter, and x 
and y are the two sample averages. The test statistic is then compared to the critical 
value for a two-sided test. The null hypothesis is rejected if 



One method used to determine the confidence level a, when several /- 
tests are conducted, is the Bonferroni method. Essentially, the Bonferroni method divides 
the original a value, in this case 0.05, by the total possible number of /-tests conducted 
[Ref. 8:p. 424]. This method offers a technique for placing a tighter restriction on the 



H 0 '-H x ~ = A, 

versus H u : ji x - fl y * A. 



x - y - A 




f 0r 1 — l all,2n-2 ‘ 
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rejection region of the null hypothesis, which would be otherwise rejected 5% of the time 
simply by chance if the original a was used. When the Mest was applied to an MOP, six 
different pair-wise tests were necessary to encompass three scenarios executed in two 
environments. Since a total of 6 comparisons were conducted, the new confidence level 
used for these Mests was 

a = a ., / 6 = 0.0083. 

b. Methods for Testing Assumptions 

Before applying the two sample Mests, the assumptions of normality and 
common variance between sample populations were considered. Two tests were 
conducted to determine if the assumption of normality was satisfied. First, Quantile- 
Quantile Plots (Q-Q Plots) for the normal distribution were plotted. The results are 
shown in Appendix F. The population is considered normally distributed if the plot 
produces a generally straight line. 

The Kolmogorov-Smirnov (KS) Goodness of Fit (GOF) was also used to 
test for the normality assumption. The test determines if a sample population follows a 
prescribed distribution. The KS GOF tests the following hypothesis: 

H o : Sample Population ~ Normal (3c,s 2 ) , versus 
H u : True cdf does not equal normal distribution for at least one sample point. 

For small sample sizes, such as 10, the KS GOF rarely rejects the null hypothesis at a 
a=0.05 level of significance. Because of this fact, the KS GOF was used as a secondary 
method to test for normality after visually inspecting the Q-Q Plots. The results of the KS 
GOF are found in Appendix G, with highlighted areas being those MOPs with p-values 
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less than 0.05, which rejects the null hypothesis. The p-value is the smallest level of 
significance at which H 0 would be rejected when a specified test procedure is used on a 
given data set [Ref. 6:p. 334], In general, the results of the test concur with the findings 
from the Q-Q Plots. 

The next assumption required for the two sample /-test is that both 
samples have a common variance. The test used to check the variance assumption was 
the F-test for common variances. The test determines if the ratio of the variances is equal 
to one. In this case, a two-sided F-test is used with a a=0.05 level of significance. The 
hypothesis tested is 



jj _ G JanusMOP 1 

LI . ~ — 1, 

o 2 

G ~ JLinkMOP 



rr G JanusMOP 

versus , H a : — 

G JLinkMOP 



* 1 . 



The results of the F-test are in Appendix H, with the highlighted portions being those 
MOPs which rejected the null hypothesis at the a=0.05 level of significance and did not 
satisfy the common variance assumption. 

For MOPs detection range, kill range, and rounds fired, the /-test was 
applied regardless of the status of the assumptions in order to provide confidence 
intervals for further analysis. The resulting confidence intervals and p-values are found in 
Appendix I. 



38 



c. 



Power of the Two Sample <-Test 



Power is defined as the probability of rejecting a false null hypothesis 
[Ref. 8:p. 79], The power of the t-test is the probability of detecting an acceptable 
difference A, in the sample means for each MOP. In terms of the null and alternative 
hypothesis, power is the probability of rejecting 

H o :A = 0, 

in favor of 



H a : A = predetermined value, 

for any true H u . 

The power of the two-sample r-test is a function of the sample sizes n and 
m, the significance level a, the population standard deviation a, and the real difference 
between the sample means A [Ref. 7 :p. 354, 394]. In this study, all scenarios were run an 
equal number of times; therefore the sample sizes were n = m = 10 for each MOP. The 
chosen significance level is a = 0.05. Using the assumption that the two populations 
from which the MOPs are drawn have a common standard deviation a, the pooled sample 
standard deviation is used to estimate a as follows: 

I 2 2 

/ S JanusMOP + S JLINKMOP , . . 

s p = , , (since n = m). 



Determining the real A, or the value of the difference between means that 
is considered significant, is rather subjective. For FASCAM kills and chemical kills, a 
10% difference would generally be acceptable between the Janus stand-alone and JLink 
results. A total of 10 units were sent through the FASCAM minefields for all scenarios. 
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resulting in a value of A = 1 for FASCAM. A total of 20 units were sent through the 
chemical artillery for scenarios 1 and 2, and 200 units (due to aggregation of infantry) 
were sent through chemical artillery for scenario 3. This results in a value of A = 2 for 
chemical artillery in scenarios 1 and 2, and A = 20 for scenario 3. The accepted 
difference in mean LERs is 0.3. This difference is based on the expected attrition of each 
scenario and the impact such a difference would make on the outcome of the battle. The 
accepted difference in the mean rounds fired is approximately 20% of the total Janus 
rounds fired for each scenario. 

The accepted difference in mean detection ranges and kill ranges is 5% of 
the predominant weapon’s capabilities. The predominant weapon is defined as the most 
widely used weapon in the scenario, in this case the tank for scenarios 1 and 2, and the 
rifle for scenario 3. The maximum detection range for the tank and rifle is six and two 
kilometers respectively, resulting in a A = 0.3 for scenarios 1 and 2, and a A = 0. 1 for 
scenario 3. The maximum kill range for the tank is four kilometers, while the rifle has a 
kill range of one kilometer. This results in a A = 0.2 for killing ranges in scenarios 1 and 
2, and A = 0.05 for scenario 3. The rationale for selecting 5% for the killing range 
follows. Assuming the tank travels at 20 kilometers per hour, then a 5% difference in 
killing range results in an additional killing range of 0.2 kilometers, or additional standoff 
time of 0.6 minutes. Since a tank is able to fire four rounds per minute, an acceptable 
difference of two additional rounds will be fired by either the Janus tank or JLink tank. 

The next step in determining the power is to calculate 4>, the non-centrality 
parameter as follows: 
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where < 7 - is the standard deviation of the sample estimator for A [Ref. 7: p. 355, 394]. 
Further, 



This equation illustrates that as sample size increases, (f> also increases, and likewise the 
power of the test. 



the calculated power values for each MOP by scenario and environment. The power is 
determined by entering the noncentral T chart at the appropriate (Rvalue, using 18 degrees 
of freedom (since two means were estimated from 20 total data points), and reading the 
corresponding power value [Ref. 7:p. 355]. So, for example, our sample of size 10 ought 
to detect a 20% difference in “rounds fired” about 54% of the time (Table 2, Scenario #1, 
SWA). The results in Table 2 show that the experiment produces power values greater 
than 50% of detecting a difference A in the sample means for most MOPs. Also, power 
values for detection ranges and kill ranges are approximately 70% or greater in most 
scenarios. Power values in the ranges noted above are generally sufficient for analysis of 
combat simulations. 




Appendix D shows the values required to calculate power. Table 2 shows 
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Scenario#!, SWA 




FASCAM 


Chem Kills 


LER 


DetRng 


Kill Rng 


Rnds 


Power 


1.00 


1.00 


.72 


1.00 


.88 


.54 


Scenario #1,HL 




FASCAM 


Chem Kills 


LER 


DetRng 


Kill Rng 


Rnds 


Power 


.31 


.81 


.54 


1.00 


.93 


.29 


Scenario #2, SWA 




FASCAM 


Chem Kills 


LER 


DetRng 


Kill Rng 


Rnds 


Power 


.28 


1.00 


.83 


1.00 


.56 


.25 


Scenario #2, HL 




FASCAM 


Chem Kills 


LER 


DetRng 


Kill Rng 


Rnds 


Power 


.40 


.88 


.62 


1.00 


.94 


.40 


Scenario #3, SWA 




FASCAM 


Chem Kills 


LER 


DetRng 


Kill Rng 


Rnds 


Power 


.40 


1.00 


.40 


1.00 


.69 


1.00 


Scenario #3, HL 




FASCAM 


Chem Kills 


LER 


DetRng 


Kill Rng 


Rnds 


Power 


.72 


1.00 


<20 


1.00 


.31 


.986 



Table 2. Power Values for MOPs 



3. Wilcoxon Test 

To substantiate the results of the two sample f-test and to test those MOPs which 
did not satisfy the r-test assumptions, the less powerful Mann-Whitney-Wilcoxon 
(Wilcoxon) nonparametric test was applied to compare the sample means. The Wilcoxon 
test uses the same two-tailed hypothesis test as the two sample Mest. The nonparametric 
test does not require that the data satisfy assumptions of normality and common variance. 
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The Wilcoxon test begins by rank-ordering all samples from both populations and 
assigning ranks 1 through m+n (20 in this case) to all values. If sample values are exactly 
equal, each value is assigned the average of the ranks that would have been assigned had 
there been no tie. The test statistic is found by first finding the sum (S) of the ranks 
assigned to the observations from the first population. The test statistic is given by 

T = S - — + ^ 

2 

The test statistic is then compared to the critical value for a two-sided test. The null 
hypothesis is rejected if 

T<0) a/2 or T > (U,. a/2 , 

where (O a/2 is the a/ 2quantile of T [Ref. 9:p. 224-226]. Conover [Ref. 9: Table 8] 

provides a table of critical values for various quantiles of T. The confidence level a is 
determined using the same Bonferroni method applied in the Mest. As with the Mest, the 
confidence level used was 

a M / 6 = 0.0083. 

C. ANALYSIS OF MOPs 

The data set used to analyze FASCAM kills, chemical kills, and LERs is found in 
Appendix A. The MOPs detection range, kill range, and rounds fired were analyzed by 
side (Appendix B) to provide a clearer interpretation of the results and additional 
information to whether JLink is accurately simulating units performing a specific type of 
mission, in this case attacking and defending. 
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As previously mentioned, the methodology of analyzing the MOPs began with the 
ANOVA test. The ANOVA considers all three factors in the experiment at all levels to 
determine if the sample mean of a particular MOP in Janus is statistically similar to the 
sample mean of the same MOP in JLink. The results of the ANOVA tests in Appendix E 
conclude that significant differences exist in the sample means between the two modes, 
except in the instance of red rounds fired. These results indicate that the multiple factors 
in the experiment cause sample means to be different when analyzed as a whole. The 
reasons for the differences are initially apparent after inspecting the raw data in 
Appendices A and B. Specifically, Janus and JLink produced significantly different 
chemical kills for every scenario, and detection ranges and kill ranges are consistently 
larger for JLink. Given these disparate inputs to the ANOVA test, one would expect the 
test to generate statistically different sample means between the two modes. To provide 
further insight to the particular differences between Janus and its distributed counterpart, 
the analysis proceeded by identifying significant interactions between factors for each 
MOP and applying the pair-wise two sample Mest and Wilcoxon test to determine where 
differences existed, leading to recommendations for improvements to JLink. 

1. FASCAM 

FASCAM was tested using the results in Table 3. 
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Summary Statistics 


Janus Stand- 


JLink vs 






Alone 


JLink 


Armored SWA 


Average 


2.7 


2 


Variance 


2.68 


1.33 




Stand Dev 


1.63 


1.15 


Armored HL 


Average 


2.9 


1.5 


Variance 


2.99 


.72 




Stand Dev 


1.73 


.85 


Armored Coalition SWA 


Average 


2.3 


1.5 


Variance 


2.68 


1.61 




Stand Dev 


1.63 


1.27 


Armored Coalition HL 


Average 


2.6 


.6 


Variance 


2.49 


.27 




Stand Dev 


1.58 


.52 


Light Infantry SWA 


Average 

Variance 


1.1 

2.54 


.2 

.4 




Stand Dev 


1.60 


.63 


Light Infantry HL 


Average 

Variance 


1 

1.11 


.6 

.27 




Stand Dev 


1.05 


.52 



Table 3. FASCAM Results 



First, the t-test assumptions were examined. Table 4 summarizes the results of 
visually inspecting the Q-Q Plots in Appendix F and the results of the KS GOF test in 
Appendix G. The highlighted portions are those areas which did not satisfy the normal 
assumption. 





Scenario#!, SWA 


Scenario #1,HL 


Scenario #2, SWA 


Mode 


Janus 


Jlink 


Janas 


Jlink 


Janus 


JLink 


Normal 


yes 


yes 


yes 


yes 


yes 


yes 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JLink 


Janus 


Jlink 


Janus 


Jlink 


Normal 


no 


no 


yes 


no 


yes 


no 



Table 4. FASCAM Normality 



Next, the F-test was used to check for common variance between the Janus and 
JLink populations. The results of the F-tests are in Appendix H. Table 5 summarizes the 
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results of the tests for normality and common variance for each scenario. The highlighted 
portions are those populations which failed either the normality assumption or the 
common variance assumption, or both. The two sample /-test was only performed on 
those Janus/JLink pairs that met the normality assumption and passed the common 
variance test. 





Scenario#!, SWA 


Scenario #1,HL 


Scenario #2, SWA 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Assumptions 


yes 


no 


yes 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Assumptions 


no 


no 


no 



Table 5. FASCAM Satisfying Normality and Common Variance 
Appendix I provides results of the pair-wise two sample /-tests, to include p- 
values and confidence intervals (Cl), and highlights those pairs which reject the null 
hypothesis. Table 6 summarizes the FASCAM results. Note that both pairs tested failed 
to reject the null hypothesis. 





Scenario#!, SWA 


Scenario #1,HL 


Scenario #2, SWA 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


fail to rej null hyp 


na 


fail to rej null hyp 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


na 


na 


na 



Table 6. FASCAM Results of Two Sample /-Test 
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The Wilcoxon test was then applied to all pair-wise comparisons of 
FASCAM. Appendix J provides the results of the Wilcoxon test, and highlights those 
pairs which reject the null hypothesis. Tables 7 summarizes the FASCAM results and 
highlights those pairs which reject the null hypothesis that the two sample averages are 
statistically similar. 





Scenario#!, SWA 


Scenario #1,HL 


Scenario #2, SWA 


Mode 


Janus 


JLink 


Janas 


JLink 


Janus 


JLink 


Results 


fail to rej null hyp 


fail to rej null hyp 


fail to rej null hyp 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


*•**&*& 

mill hyp 


fail to rej null hyp 


fail to rej null hyp 



Table 7. FASCAM Results of Wilcoxon Test 



2. Chemical Kills 

The chemical kills MOP was analyzed using the results in Table 8. 
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Summary Statistics 


Janus Stand- 


JLink vs 






Alone 


JLink 


Armored SWA 


Average 


4.4 


.3 


Variance 


.49 


.23 




Stand Dev 


.70 


.48 


Armored HL 


Average 


12.3 


2.5 


Variance 


3.12 


1.39 




Stand Dev 


1.77 


1.18 


Armored Coalition SWA 


Average 


3.8 


.1 




Variance 


.4 


.1 




Stand Dev 


.63 


.32 


Armored Coalition HL 


Average 


12.3 


2.9 


Variance 


2.68 


.99 




Stand Dev 


1.64 


.99 


Light Infantry SWA 


Average 

Variance 


9.9 

10.1 


.6 

1.6 




Stand Dev 


3.18 


1.26 


Light Infantry HL 


Average 

Variance 


69.6 

1.6 


36.6 

77.6 




Stand Dev 


1.26 


8.81 



Table 8. Chemical Results 



The analysis applied the two sample r-test to identify the specific scenarios which 
produced statistically different chemical kills between Janus and JLink. First, the 
assumptions were checked in the same manner as with FASCAM. Normality was 
determined based on the visual inspection of the Q-Q Plots and the results of the KS GOF 
test. Then, common variance was determined based on the results of the F-test. Table 9 
summarizes the results of the tests for normality and common variance for each scenario. 
The highlighted portions are those populations which failed either the normality 
assumption or the common variance assumption, or both. Again, the two sample r-test 
was only performed on those Janus/JLink pairs that met the normality assumption and 
passed the common variance test. 
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Scenario #1, SWA 


Scenario #1,HL 


Scenario #2, SWA 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JUnk 


Assumptions 


no 


yes 


no 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Assumptions 


no 


no 


no 



Table 9. Chemical Kills Satisfying Normality and Common Variance 

The t-test results in Appendix I show that the one pair-wise comparison tested 
rejected the null hypothesis that the average number of chemical kills between Janus and 
JLink are statistically similar in scenario #1, HL. The result is summarized (for 
consistency) in Table 10. 





Scenario#!, SWA 


Scenario #1,HL 


Scenario #2, SWA 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


na 


mj null hyp 


na 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


na 


na 


na 



Table 10. Chemical Results of Two Sample f-Test 



The Wilcoxon test was then applied to all pair-wise comparisons of chemical 
kills. Appendix J provides the results of the Wilcoxon test, and highlights those pairs 
which reject the null hypothesis. Tables 1 1 summarizes the chemical results and 
highlight those pairs which reject the null hypothesis that the two sample averages are 
statistically similar. 



49 





Scenario #1, SWA 


Scenario#!, HI. 


Scenario #2, SWA 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


Jlink 


Results 


rej null hyp 


rej null hyp 


rej null hyp 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


Jlink 


Janus 


JLink 


Janus 


Jlink 


Results 


rej null hyp 


rej null hyp 


rej null hyp 



Table 11. Chemical Results of Wilcoxon Test 



3. Detection Range (Blue/Red) 



The detection range MOP was analyzed using the results in Tables 12 and 13. As 



previously mentioned, the data for detection range was analyzed separately by blue and 
red force. 





Summary Statistics 


Janus Stand- 


JLink vs 






Alone 


JLink 


Armored SWA 


Average 


3.689 


4.171 


Variance 


.010 


.005 




Stand Dev 


.098 


.072 


Armored HL 


Average 


3.330 


3.951 


Variance 


.005 


.012 




Stand Dev 


.072 


.107 


Armored Coalition SWA 


Average 


3.619 


4.073 


Variance 


.033 


.011 




Stand Dev 


.181 


.104 


Armored Coalition HL 


Average 


3.362 


3.948 


Variance 


.009 


.006 




Stand Dev 


.096 


.074 


Light Infantry SWA 


Average 

Variance 


2.067 

.002 


2.075 

.001 




Stand Dev 


.040 


.027 


Light Infantry HL 


Average 

Variance 


1.648 

.001 


1.719 

.001 




Stand Dev 


.033 


.036 



Table 12. Blue Detection Range Results 
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Summary Statistics 


Janus Stand- 


JLink vs 






Alone 


JLink 


Armored SWA 


Average 


2.470 


3.389 


Variance 


.027 


.021 




Stand Dev 


.165 


.146 


Armored HL 


Average 


2.000 


2.882 


Variance 


.012 


.034 




Stand Dev 


.108 


.183 


Armored Coalition SWA 


Average 

Variance 


2.212 

.011 


3.242 

.027 




Stand Dev 


.103 


.163 


Armored Coalition HL 


Average 


1.888 


2.624 


Variance 


.007 


.054 




Stand Dev 


.081 


.232 


Light Infantry SWA 


Average 

Variance 


1.791 

.001 


2.008 

.001 




Stand Dev 


.030 


.030 


Light Infantry HL 


Average 

Variance 


1.559 

.001 


1.672 

.002 




Stand Dev 


.027 


.041 



Table 13. Red Detection Range Results 



The ANOVA interaction plots in Appendix E for both blue and red detections show 
significant interaction between the scenario and the mode, illustrated by the larger gap in 
detection differences for scenarios 1 and 2 as compared to the detection difference for 
scenario 3. Such interaction is expected given that scenarios 1 and 2 used weapon 
systems with detection capability up to six kilometers, whereas scenario 3 used infantry 
units with detection capability of roughly two kilometers. 

Testing for Mest assumptions showed that all detection populations satisfied the 
normality assumption and only the red detection range in scenario # 2, HL rejected the 
assumption of common variance. Tables 14 and 15 show the summarized results of the t- 
test for blue and red detection ranges, respectively. 



51 






Scenario #1, SWA 


Scenario#!, HI, ^ 


Scenario #2, SWA 


Mode 


Janas 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


iq null hyp 


rq null hyp 


iq null hyp 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


iq null hyp 


fail to rej null hyp 


rq' null hyp 



Table 14. Blue Detection Range Results of /-Test 





Scenario #1, SWA 


Scenario#!, HL 


Scenario #2, SWA 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


iq null hyp 


iq null hyp 


rej null hyp 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


na 


iq null hyp 


rejnullhyp 



Table 15. Red Detection Range Results of /-Test 



The Wilcoxon test was also applied to all pair-wise comparisons of detection 
ranges. Tables 16 and 17 summarize the results from Appendix J and highlight those 
pairs which reject the null hypothesis that the two sample averages are statistically 
similar. 
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Scenario#!, SWA 


Scenario #1,HL 


Scenario #2, SWA 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


ig null hyp 


ig null hyp 


ig null hyp 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


ig null hyp 


fail to rej null hyp 


xg null hyp 



Table 16. Blue Detection Range Results of Wilcoxon Test 





Scenario #1, SWA 


Scenario #1, HL 


Scenario #2, SWA 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


xg null hyp 


ig null hyp 


rej null hyp 






Scenario #2, 1 IL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


xg null hyp 


ig null hyp 


xg null hyp 



Table 17. Red Detection Range Results of Wilcoxon Test 



4. Kill Ranges (Blue/Red) 

The kill range MOP was analyzed using the data in Tables 19 and 20. The data 
for kill ranges was analyzed separately by blue and red force. 
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Summary Statistics 


Janus Stand- 


JLink vs 


- 




Alone 


JLink 


Armored SWA 


Average 


2.074 


2.394 


Variance 


.123 


.006 




Stand Dev 


.350 


.076 


Armored HL 


Average 


1.955 


2.556 


Variance 


.045 


.061 




Stand Dev 


.212 


.246 


Armored Coalition SWA 


Average 


1.987 


2.325 


Variance 


.136 


.216 




Stand Dev 


.369 


.465 


Armored Coalition HL 


Average 


1.831 


2.145 


Variance 


.050 


.120 




Stand Dev 


.224 


.346 


Light Infantry SWA 


Average 

Variance 


1.477 

.005 


1.626 

.001 




Stand Dev 


.072 


.027 


Light Infantry HL 


Average 

Variance 


1.301 

.003 


1.375 

.001 




Stand Dev 


.055 


.023 



Table 18. Blue Kill Range Results 





Summary Statistics 


Janus Stand- 


JLink vs 






Alone 


JLink 


Armored SWA 


Average 


2.847 


3.100 


Variance 


.033 


.061 




Stand Dev 


.181 


.246 


Armored HL 


Average 


2.607 


3.00 


Variance 


.016 


.007 




Stand Dev 


.127 


.084 


Armored Coalition SWA 


Average 


3.00 


3.00 


Variance 


.025 


.065 




Stand Dev 


.158 


.255 


Armored Coalition HL 


Average 


2.626 


3.012 


Variance 


.018 


.009 




Stand Dev 


.135 


.095 


Light Infantry SWA 


Average 

Variance 


1.313 

.005 


1.369 

.004 




Stand Dev 


.067 


.067 


Light Infantry HL 


Average 

Variance 


1.317 

.004 


1.455 

.018 




Stand Dev 


.067 


.133 



Table 19. Red Kill Range Results 

The ANOVA interaction plots illustrate a significant interaction between the environment 
and the mode for red kill ranges. This interaction is interpreted as, not only are the mean 
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red kill ranges different between Janus and JLink, the differences are greater in HL terrain 
than SWA terrain. 

Testing for f-test assumptions showed that all red kill range populations satisfied 
both the normality and common variance assumptions. Table 20 summarizes which 
scenarios satisfied the assumptions for blue kill ranges. 





Scenario #1, SWA 


Scenario #1, HL 


Scenario #2, SWA 


Mode 


Janus 


JUnk 


Janus 


JLink 


Janus 


JLink 


Assumptions 


no 


yes 


no 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Assumptions 


yes 


no; 


no 



Table 20. Blue Kill Ranges Satisfying Normality and Common Variance 

Tables 21 and 22 summarize the results from Appendix I of the two sample r-test 
for blue and red kill ranges, respectively. 





Scenario#!, SWA 


Scenario #1,HL 


Scenario #2, SWA 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


na 


rgnuflbyp 


na 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


fail to rej null hyp 


na 


na 



Table 21. Blue Kill Range Results of t-Test 
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Scenario #1, SWA 


Scenario #1,HL 


Scenario #2, SWA 


Mode 


Janus 


JUnk 


Janus 


JUnk 


Janus 


JUnk 


Results 


fail to rej null hyp 


rej null hyp 


fail to rej null hyp 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JUnk 


Janus 


JUnk 


Janus 


JUnk 


Results 


tq null hyp 


fail to rej null hyp 


fail to iq null hyp 



Table 22. Red Kill Range Results of f-Test 



The Wilcoxon test was then applied to all pair-wise comparisons of kill ranges. 
Tables 23 and 24 summarize the results from Appendix J and highlight those pairs which 
reject the null hypothesis that the two sample averages are statistically similar. 





Scenario#!, SWA 


Scenario #1,HL 


Scenario #2, SWA 


Mode 


Janus 


JUnk 


Janus 


JUnk 


Janus 


JUnk 


Results 


fail to rej null hyp 


rej null hyp 


fail to rej null hyp 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JUnk 


Janus 


JUnk 


Janus 


JUnk 


Results 


fail to rej null hyp 


tq null hyp 


rej null hyp 



Table 23. Blue Kill Range Results of Wilcoxon Test 
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Scenario#!, SWA 


Scenario#!, 1EL 


Scenario #2, SWA 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


fail to rej null hyp 


rej null hyp 


fail to rej null hyp 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


reg null hyp 


fail to rej null hyp 


fail to rej null hyp 



Table 24. Red Kill Range Results of Wilcoxon Test 



5. Rounds Fired (Blue/Red) 

The rounds fired MOP was analyzed using the data in Tables 25 and 26. The data 
for rounds fired was analyzed separately by blue and red force. 





Summary Statistics 


Janus Stand- 
Alone 


JLink vs 
JLink 


Armored SWA 


Average 
Variance 
Stand Dev 


85.3 

1292.23 

35.95 


32.8 
101.73 

10.09 


Armored HL 


Average 
Variance 
Stand Dev 


112.3 

1360.01 

36.88 


58.1 

40.32 

6.35 


Armored Coalition SWA 


Average 
Variance 
Stand Dev 


56.5 

176.5 
13.29 


35.3 

206.01 

14.35 


Armored Coalition HL 


Average 
Variance 
Stand Dev 


71.6 

401.6 
20.04 


73.6 

424.04 

20.59 


Light Infantry SWA 


Average 
Variance 
Stand Dev 


969.6 

63081.38 

251.16 


621.3 

1896.46 

43.55 


Light Infantry HL 


Average 
Variance 
Stand Dev 


1347.6 

93197.38 

305.28 


1341.5 

31205.83 

176.65 



Table 25. Blue Rounds Fired Results 
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Summary Statistics 


Janus Stand- 


JLink vs 






Alone 


JLink 


Armored SWA 


Average 


110.1 


195.3 


Variance 


1215.43 


983.79 




Stand Dev 


34.86 


31.37 


Armored HL 


Average 


531.9 


323.8 


Variance 


75729.21 


6535.07 




Stand Dev 


275.19 


80.84 


Armored Coalition SWA 


Average 


144 


165.7 


Variance 


6945.78 


918.46 




Stand Dev 


83.34 


30.31 


Armored Coalition HL 


Average 


523.4 


411.9 


Variance 


19137.16 


18999.88 




Stand Dev 


138.34 


137.84 


Light Infantry SWA 


Average 

Variance 


1626 

11621.11 


1619 

4510 




Stand Dev 


107.80 


67.16 


Light Infantry HL 


Average 

Variance 


1365.5 

64852.5 


1594 

53604.44 




Stand Dev 


254.66 


231.53 



Table 26. Red Rounds Fired Results 



The results of the ANOVA test in Appendix E show that the average red rounds fired in 
JLink is statistically similar to the average red rounds fired in Janus, as evidenced by the 
p- value of .96 for Mode. Because of this result, no further statistical tests were conducted 
on the red rounds fired. Also, the results of the ANOVA interaction plots illustrate 
interaction between the environment and mode for blue rounds fired. The difference 
between Janus and JLink in the mean blue rounds fired in SWA is significantly larger 
than the mean blue rounds fired in HL. 

Next, the two sample f-test was applied to all pairs of blue rounds fired. Results 
from testing for normality and common variance are summarized in Table 27. 
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Scenario#!, SWA 


Scenario #1,HL 


Scenario #2, SWA 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Assumptions 


no 


no 


no 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JUnk 


Janus 


JLink 


Janus 


JLink 


Assumptions 


yes 


no 


yes 



Table 27. Blue Rounds Fired Satisfying Normality and Common Variance 

Table 28 summarizes the results from Appendix I of the two sample /-test for blue 
rounds fired. 





Scenario#!, SWA 


Scenario #1,HL 


Scenario #2, SWA 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


na 


na 


na 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JLink 


Janus 


JUnk 


Janus 


JLink 


Results 


fail to rej null hyp 


na 


fail to rej null hyp 



Table 28. Blue Rounds Fired Results of /-Test 



The Wilcoxon test was then applied to all pair-wise comparisons of blue rounds 
fired. Table 29 summarizes the results from Appendix J and highlights those pairs which 
reject the null hypothesis that the two sample averages are statistically similar. 
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Scenario #1, SWA 


Scenario #1, HL 


Scenario #2, SWA 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


rej null hyp 


rej null hyp 


fail to rej null hyp 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


fail to rej null hyp 


rej null hyp 


fail to rej null hyp 



Table 29. Blue Rounds Fired Results of Wilcoxon Test 



6. Loss Exchange Ratio (LER) 

The LER MOP was analyzed using the results in Table 30. 





Summary Statistics 


Janus Stand- 
Alone 


JLink vs 
JLink 


Armored SWA 


Average 
Variance 
Stand Dev 


2.182 

.072 

.269 


1.161 

.074 

.273 


Armored HL 


Average 
Variance 
Stand Dev 


1.659 

.156 

.395 


1.505 

.049 

.221 


Armored Coalition SWA 


Average 
Variance 
Stand Dev 


1.377 

.025 

.160 


1.174 

.070 

.265 


Armored Coalition HL- 


Average 
Variance 
Stand Dev 


1.489 

.040 

.201 


1.835 

.122 

.350 


Light Infantry SWA 


Average 
Variance 
Stand Dev 


1.557 

.254 

.504 


.937 

.011 

.104 


Light Infantry HL 


Average 
Variance 
Stand Dev 


3.112 

1.004 

1.002 


2.36 

.436 

.660 



Table 30. LER Results 



The analysis applied the two sample t-test to identify the specific scenarios which 



produced statistically different LERs between Janus and JLink. First, the /-test 



assumptions were checked in the same manner as before. The tests revealed that all 
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scenarios satisfied both the normality and common variance assumption except scenario 



#3, SWA. The results of the r-tests are summarized in Table 31 . 





Scenario#!, SWA 


Scenario #1,HL 


Scenario #2, SWA 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


rej null hyp 


fail to rej null hyp 


fail to rej null hyp 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


fail to rej null hyp 


na 


fail to rej null hyp 



Table 31. LER Results of Two Sample <-Test 



The Wilcoxon test was then applied to all pair-wise comparisons of LERs. Tables 
32 summarizes the LER test results and highlights those pairs which reject the null 
hypothesis that the two sample averages are statistically similar. 





Scenario#!, SWA 


Scenario #1,HL 


Scenario #2, SWA 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


rej null hyp 


fail to rej null hyp 


fail to rej null hyp 






Scenario #2, HL 


Scenario #3, SWA 


Scenario #3, HL 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Results 


fail to rej null hyp 


rejnuBhyp 


fail to rej null hyp 



Table 32. LER Results of Wilcoxon Test 



D. VARIABILITY WITHIN JLINK 

The data in Appendix C are the results of 10 JLink runs, using the same scenarios 
and environments as described before, except that the same random number seed was 
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used for all runs. The reason for conducting these runs was to determine the variance 
within the JLink system and compare that variance to runs using different random number 
seeds. In essence, this test isolated the variance contributed by JLink alone. 

Inspecting the variances in Appendix C, and comparing them to the variances of 
the runs using 10 different random number seeds in Appendix A, we see that the JLink 
variance using the same seed was generally slightly smaller than the variance in Appendix 
A. This finding indicates that most of the variability is generated within the runs, instead 
of from the changes in the random number seed. 
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V. ANALYSIS OF RESULTS 



The results of the statistical tests from Chapter IV provide a firm basis for 
assessing the differences between the JLink system and its parent simulation, Janus. This 
chapter will analyze the results of the ANOVA test, two sample Mest, and Wilcoxon test 
and offer insights to the causes of these differences. Since the ANOVA test rejected the 
null hypothesis that the averages were statistically similar between Janus and JLink for all 
MOPs except red rounds fired, the analysis will focus primarily on where the differences 
occurred based on the results of the Mest and Wilcoxon test. The analysis includes 
possible causes for unexpected results and provides recommendations to resolve those 
issues derived from such results. The analysis of the results is presented by individual 
MOP. 



A. FASCAM KILLS 

The results from the two sample Mest and Wilcoxon test rejected the null 
hypothesis that the average FASCAM kills are the same between Janus stand-alone and 
JLink in only one instance, for scenario #2, HL. This would lead one to believe that 
JLink is accurately representing FASCAM under various conditions. Also, four out of six 
times Jlink produced average results which were within the accepted tolerance of A = 1 
kill. However, two issues must be addressed before drawing a final conclusion about 
JLink FASCAM: the location of the FASCAM minefield and the trend in the data 
between Janus and JLink. 
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As mentioned in Chapter III, paragraph C. 2, the player fires FASCAM from an 
artillery platform after the scenario begins. To remove aim-error variability due to human 
interaction, all runs were “pucked” to provide a common firing point between scenarios. 
Ballistic errors in azimuth and elevation are another source of location variability when 
firing from an artillery platform. Janus simulates these ballistic errors by drawing the x 
and y coordinates of the round’s impact point from a normal distribution. So, in order to 
ensure that a FASCAM minefield is placed precisely in the same location in different 
runs, the runs must be “pucked” under almost identical conditions to ensure reasonable 
azimuth and elevation errors. The experiment satisfied the first condition, but due to 
JLink architecture and different random number generators, dispersion caused by the 
latter condition could not be completely controlled. 

To reduce the dispersion of the minefields, FASCAM was fired in a 
relatively small geographic location, compared to the entire scenario. By firing five 
minefields per run, for 10 runs, in a relatively small location, one would expect the effect 
of minefield dispersion to be negligible over the entire experiment. However, after 
replaying some of the scenarios on the Janus Analyst Workstation and comparing the 
locations of the minefields between modes, it was evident that minefield dispersion 
would contribute to dissimilar FASCAM kills in Janus and JLink. Recall that blue forces 
fired FASCAM at specified red units traveling on a narrow route. Suppose that four of 
the five FASCAM minefields fired in Janus landed on the designated route whereas only 
three JLink minefields landed on the route, due to ballistic error. Under these 
circumstances, the experiment would not isolating the effects of the FASCAM 
minefields. 
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Another issue regarding the FASCAM results is the consistent trend towards 
greater FASCAM kills in Janus as compared to JLink. Such a trend forces the analyst to 
proceed with caution when drawing conclusions from the statistical tests. Although the 
tests indicate that the FASCAM results are statistically similar between modes (except in 
one instance), such a trend may indicate that the JLink FASCAM does not produce the 
same number of kills per mine as Janus FASCAM. 

Given the results of the tests, the current configuration of JLink provides a fair 
representation of FASCAM as compared to Janus. Although there appears to be a trend 
in average FASCAM kills favoring Janus, the significance of the trend is diminished 
somewhat by the fact that one out of every six runs produced greater JLink FASCAM 
kills than Janus. The trend in average kills, however, does merit further investigation. 
Additional testing, isolating the effects of FASCAM, is necessary to determine if the 
JLink FASCAM is as lethal as Janus FASCAM. 

B. CHEMICAL KILLS 

In all cases, the Mest and the Wilcoxon test rejected the null hypothesis 
that the average chemical kills are the same between Janus stand-alone and JLink. 
Clearly, JLink’ s current configuration does not accurately replicate the effects of Janus 
chemical artillery. 

In an attempt to identify the cause of the disparate test results and rectify 
the shortcoming, the JLink delivery method shown in Figure 2 was investigated. As 
aforementioned, Janus calculates artillery by volley as opposed to individual rounds, as 
dictated by the DIS standard. Calculating the artillery by volley results in a single large 
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chemical cloud which stays together for a much longer period of time as opposed to 
individual clouds which dissipate quickly and cover less area. Since the smaller clouds 
dissipate quickly, entities in contact are affected less by the smaller chemical clouds than 
by the larger aggregated cloud. 

In order to correct this shortcoming, the World Modeler software was rewritten so 
that whenever a user participates in a JLink to JLink game, all chemical artillery will be 
fired and portrayed by volley as opposed to individual rounds. After making the 
corrections in the code, 10 additional runs were performed using the SWA armored 
scenario. The results of the scenario were more in line with what was originally 
expected. 

The ten runs of both Janus and JLink yielded an average of 10.3 and 10.5 
chemical kills respectively (Table 33). Because the data satisfied the assumptions of 
normality and common variance with a Q-Q Plot and F-test respectively, a two-sample t- 
test was also conducted. Since the generated test statistic 0.7356 is less than the critical 
value of 2.101, the conclusion is that the means do not differ significantly. 



RUN# 


Chemical Kills 


Janus 


JLink 


1 


9 


9 


2 


11 


11 


3 


12 


11 


4 


9 


11 


5 


9 


11 


6 


11 


12 


7 


11 


10 


8 


8 


11 


9 


11 


8 


10 


12 


11 


XBAR 


10.3 


10.5 


VAR 


2.011 


1.389 


STDE 


1.418 


1.179 



Table 33. Chemical Kills for Volley Fire 
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These results support the finding that JLink accurately represents chemical 
artillery in a SWA armored scenario. Additional testing using other scenarios is 
necessary to conclude that JLink accurately portrays chemical artillery under varied 
conditions. The results show that chemical rounds must be fired in volleys, as opposed to 
individual rounds, whenever a user plays a JLink to JLink game. Software upgrades will 
include a toggle transparent to the user so that JLink automatically sends out volleys as 
opposed to individual rounds whenever a user indicates that the other participating DIS 
simulation is another JLink. 

C. DETECTION RANGES (BLUE/RED) 

The next MOP analyzed was detection range, analyzed separately by blue force 
and red force, using the data from Appendix B. (The headings “Det Ranges Blue/Red” in 
Appendix B refer to the those units doing the detecting). As mentioned previously, 
separating the detection ranges by side offers a clearer interpretation of the results and 
additional information to whether JLink is accurately simulating units performing a 
specific type of mission, in this case attacking and defending. 

The two-sample Mest and the Wilcoxon test rejected the null hypothesis that the 
average blue and red detection ranges are similar for every scenario except for the blue 
detection ranges in scenario #3, SWA. 

Next, the thesis compared the accepted A = 0.3 kilometer tolerance for detection 
ranges in scenarios 1 and 2, and A = 0. 1 kilometers for scenario 3, to the confidence 
intervals from Appendix I, to determine if any detection ranges were within acceptable 
tolerance. The comparison concluded that none of the detection ranges were within the 
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acceptable tolerance, with the one exception noted above. (Figure 7 illustrates the 
method for applying the A value to the confidence interval for scenario #3, HL, blue 
detection range). Additionally, a consistent trend in the detection data surfaced; the JLink 
detection ranges were always larger than the Janus detection ranges, for both the blue and 
red forces. 




Figure 7. Confidence Intervals Versus Acceptance Range 

The next step was to identify the cause of the disparate results and the trend in the 
data. The first issue investigated was the passing of the entity’s z-coordinate. If the 
entity’s z-coordinate is not passed properly to JLink, or interpreted properly by JLink, the 
entity could be simulated above ground level, resulting in better line of sight and 
extended detection ranges. This hypothesis turned out to be incorrect, however, because 
JLink can determine which entities are ground entities. When the information about a 
ground entity is passed to JLink, JLink assigns the entity the x and y coordinates passed 
from Janus and places the entity on the ground. The ground entity then uses the ground z- 
coordinate to perform line of sight calculations. 
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The second proposed solution focused on the fact that something in the battlefield 
environment was affecting the detection ranges. The issue of artillery delivered smoke 
and munitions was suggested as a possible cause. All scenarios used large volumes of 
artillery smoke and artillery High Explosive (HE) rounds. As discussed in the analysis of 
chemical kills, Janus and JLink simulated artillery delivery methods differently in this 
experiment. If the dimension of the chemical cloud could affect the number of chemical 
kills, then the dimensions of any cloud generated from an artillery munition may also 
affect the range at which an entity can detect through the cloud. 

To test the proposed solution, the SWA armored scenario was re-run, excluding 
all artillery from both sides. Thus, in this case there were no clouds resulting from 
artillery smoke or HE rounds on the battlefield. The hypothesis was that the artillery 
simulated as a volley (Janus) led to decreased visibility, and shorter detection ranges, than 
artillery simulated as individual rounds (JLink). The scenario was then run 10 times in 
both the Janus and JLINK modes, using the same random number seeds. The results 
yielded an average detection range for red Janus entities of 2.532 kilometers, while the 
average detection range for the red JLINK entities was 2.463 kilometers. The average 
detection range for blue Janus entities was 3.885 versus 3.767 for the blue JLINK entities. 
Table 34 depicts the actual ranges for each run. The assumption of normality was 
satisfied, but the F-Test concluded that the variances for the populations were not the 
same for the blue Janus and blue JLink detection ranges. The variances were the same, 
however, for the red Janus and red JLink detection ranges. Finally, a two-sample Mest 
showed that in both cases, the means for the populations were not significantly different. 
This analysis provides a basis for concluding that the different artillery delivery methods 
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contributed to the disparate detection results. Further, this thesis concludes that the 



volley method used by Janus to deliver artillery munitions generates shorter detection 



ranges than the individual round method due to the dimensions and dissipation rate of the 



cloud. 



Currently, the software has not been rewritten in JLink to fire artillery using the 



same method as Janus. After the code is rewritten, several additional experiments will be 



done to ensure that the artillery is being accurately played using the volley method, and 



that this method generates similar detection ranges between Janus and JLink. 





Det Ranges Blue 


Det Ranges Red 


RUN# 


Janus 


JLink 


Janus 


JLink 


1 


3.863 


3.785 


2.294 


2.711 


2 


3.940 


3.753 


2.513 


2.386 


3 


3.846 


3.846 


2.541 


2.541 


4 


3.778 


3.877 


2.408 


2.719 


5 


3.918 


3.918 


2.890 


2.889 


6 


4.017 


3.616 


2.912 


2.166 


7 


3.863 


3.567 


2.294 


2.319 


8 


3.912 


4.021 


2.715 


2.512 


9 


3.856 


3.660 


2.377 


2.275 


10 


3.853 


3.631 


2.377 


2.119 


XBAR 


3.885 


3.767 


2.532 


2.464 


VAR 


0.004 


0.022 


0.054 


0.065 


STDEV 


0.065 


0.149 


0.232 


0.255 


r-Test Stat 


0.040 


.515 


Crit. Value 


2.101 


2.101 



Table 34. Detection Ranges Without Artillery 



D. KILL RANGES (BLUE/RED) 

The next MOP analyzed was kill range, analyzed separately by blue force and red 
force, using the data from Appendix B. (The headings “Kill Ranges Blue/Red” in 



Appendix B refer to the those units being killed). The results of the two-sample f-test and 
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the Wilcoxon test rejected the null hypothesis that the average kill ranges (blue and red) 
are same between modes in six instances : 

1. Blue Kill Ranges in Scenario #1, SWA 

2. Blue Kill Ranges in Scenario #1, HL 

3. Red Kill Ranges in Scenario #1 , HL 

4. Red Kill Ranges in Scenario #2, HL 

5. Blue Kill Ranges in Scenario #3, SWA 

6. Blue Kill Ranges in Scenario #3, HL 

Next, the acceptable tolerance range of A = 0.2 kilometers for scenarios 1 and 2, 
and A = 0.05 kilometers for scenario 3, was compared to the kill range CIs in Appendix I 
for those scenarios which rejected the null hypothesis. The results of the comparison 
were that none of the CIs fell within the acceptable tolerance. This finding can be 
interpreted by stating that the mean kill ranges in Janus and JLink in these scenarios are 
not the same, and indeed so dissimilar that the difference in the means falls outside the 
acceptable tolerance. 

As with the detection ranges data, a consistent trend occurred in the kill range 
data; the kill ranges for JLink were consistently greater than the kill ranges for Janus. 

This result is not surprising in light of the similar trend in detection ranges. There is a 
strong likelihood that if an entity can detect at further range, then it can acquire and target 
at greater distances, and subsequently kill at longer ranges. This reasoning is based on 
the method by which a Janus entity performs the engagement sequence. After conducting 
a line of sight calculation, Janus determines the target’s location and identity. Once 
identified, Janus determines the target’s priority and places it on the target list. The target 



71 



list for each Janus entity determines which targets will be engaged and in what order. 
Janus then acquires the target and engages. The engagements may, or may not, lead to a 
kill. This entire sequence, however, stems from the initial detection. Therefore, if the 
detection ranges are statistically similar, then the engagement sequences in both Janus 
and JLink may be in agreement. 

The link between detection ranges and kill ranges focused the investigation into 
resolving the disparate kill range issue. As mentioned in the previous section, further 
analysis of the detection ranges, without firing artillery, has begun using the armored 
scenario in SWA. This ongoing investigation will also analyze the kill ranges to verify 
whether the kill ranges were indeed affected by the detection ranges. The assumption is 
that if the detection range issue is resolved, the kill ranges between Janus and JLink will 
become more similar. Presently, this hypothesis is being tested. 

The last result analyzed for kill ranges is the evidence of interaction between the 
environment and mode for mean red kill ranges. The interaction plot in Appendix E 
shows that the red force has the advantage in JLink of greater kill ranges, but the 
advantage is even more pronounced when fighting in HL terrain as compared to SWA 
terrain. The assumption that greater detection ranges in JLink lead to greater kill ranges 
in JLink accounts for part of the overall advantage. However, all other factors being 
equal, one would expect the JLink kill range advantage to be consistent between 
scenarios since the JLink detection range advantage is consistent between scenarios. 
Because there is a larger red kill range difference in the HL as compared to SWA, further 
investigation is needed to identify the factor causing the difference. 
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One likely factor causing the difference in red kill ranges between environments 
and mode is the different terrain features of HL and SWA. Terrain features affect line of 
sight, which then affect detections, and subsequently the entire engagement process 
leading to rounds fired. Further testing should focus on this factor by simulating battalion 
size forces attacking under these two different environments. The issue of detection 
ranges should be resolved first, however, in order to isolate the factors causing 
differences in kill ranges. 

Overall, JLink’s current configuration returns kill ranges which are statistically 
similar to those in Janus in six of 12 cases. These results indicate that JLink provides a 
setting which generally replicates the kill ranges of different types of units fighting in 
different environments. In particular, JLink appears quite accurate in producing 
comparable kill ranges for armored coalition scenarios fighting on the SWA terrain, and 
attacking light infantry forces fighting on both HL and SWA terrain. These results also 
indicate that the JLink battlefield produces fairly accurate kill ranges for units conducting 
different types of missions, in this case an attacking red force against a defending blue 
force. Once further testing is complete, and the issue of detection range is resolved, one 
would expect JLink to produce kill ranges even closer to those of Janus. 

E. ROUNDS FIRED (BLUE/RED) 

The next MOP analyzed was rounds fired, analyzed separately by blue force and 
red force, using the data from Appendix B. The ANOVA returned a p- value of .96 for 
red rounds fired, which indicates that JLink is providing a battlefield setting which 
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replicates the average number of rounds fired comparable to Janus for a battalion-size 
armored or infantry force in the offense, under different environmental conditions. 

The results of the two-sample Mest and the Wilcoxon test for blue rounds fired 
were mutually supporting, although the Mest was not performed on several data sets 
because they did not satisfy Mest assumptions, primarily due to wide differences in 
variance between populations. The tests rejected the null hypothesis that the average blue 
rounds fired are the same between modes in four instances : 

1. Blue Rounds Fired in Scenario #1, SWA 

2. Blue Rounds Fired in Scenario #1, HL 

3. Blue Rounds Fired in Scenario #2, SWA 

4. Blue Rounds Fired in Scenario #3, SWA 

The ANOVA supports these findings as indicated by the F-values for Mode in Appendix 
E. The F - value of 1 1.929 for the blue rounds fired is interpreted as, the average blue 
rounds fired among all Janus scenarios are statistically different than the average blue 
rounds fired among all JLink scenarios. 

The interaction between environment and mode for blue rounds fired indicates 
that the difference in blue rounds fired between Janus and JLink is even greater when 
placed in the SWA environment. As mentioned previously, investigation into the cause 
of this interaction must focus on the competing terrain of HL and SWA. Terrain affects 
line of sight and ultimately the engagement sequence and rounds fired. Once the 
detection range issue is resolved, further testing should focus on simulating units in the 
defense under different environments. 
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Next, the four scenarios which rejected the null hypothesis were analyzed to 
determine if the differences in the number of rounds fired between Janus and JLink 
satisfied the accepted tolerance. Recall that the accepted tolerance for rounds fired was 
A = 10% of the total Janus rounds fired. Applying the method illustrated in Figure 7 of 
comparing the MOP’s confidence interval to the determined A - value resulted in none of 
the four being within acceptable tolerance. 

Based on the results of the statistical tests for rounds fired, JLink provides an 
accurate replication of the battlefield that generates an average number of rounds fired 
comparable to Janus for a battalion-size unit attacking under different environmental 
conditions. JLink also provides a fare replication of the battlefield that generates similar 
average rounds fired for a company-size unit defending in HL terrain. 

Further, in reference to the discussion of the engagement process in paragraph D 
above, the number of rounds fired could become more similar between Janus and Jlink 
once the detection range and kill range issues are resolved. Since JLink consistently had 
greater detection ranges and kill ranges than Janus in this experiment, one would 
conclude that the number of rounds fired were affected by the disparity given that all 
three MOPs are linked in the engagement process. The ongoing experiment of the 
armored scenario in SWA, stripped of all artillery munitions, will be a valuable study to 
analyze the effect that accurate detection ranges have on rounds fired, especially since 
both the blue and red forces in the armored scenario in SWA appeared to have different 
averages of numbers of rounds fired. 
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F. 



LOSS EXCHANGE RATIO (LER) 



The LER is the ratio of red casualties to blue casualties, and offers a quantitative 
method to assess the overall outcome of a battle. Given that the factors affecting the ratio 
are accurately portrayed, LERs determined to be statistically similar provide strong 
supporting evidence that JLink is accurately portraying the entire scenario. In this case, 
both the Mest and Wilcoxon test failed to reject the null hypothesis in four of the six 
scenarios when testing the LERs. This indicates that the Janus and JLink LERs for these 
four scenarios are similar. In scenario #1, SWA and scenario #3, SWA, the hypothesis of 
equal LERs was rejected. Although, on four of six occasions, the null hypothesis was not 
rejected, an analyst must be cautious to accept the conclusion that JLink is accurately 
portraying the overall battle under these four conditions based on these results. 

The question posed then is, why are two of the six not statistically similar? The 
LER is a function of several other factors, primarily kills and detections. Previous 
analysis has shown that kills due to chemical artillery were not accurately portrayed by 
JLink during the experiment. Chemical kills is a factor of the LER, and inaccuracies with 
simulating those kills will adversely affect the LER for that scenario. Additionally, 
discussion of the detection ranges concluded that JLink did not satisfactorily replicate the 
detection ranges in this experiment, which is the another major contributing factor of the 
LER. The chemical kills and detection range issues alone could lead to dissimilar LERs 
between Janus and JLink. 

Given that at least two inputs to the LER equation may not have been accurate, it 
is reassuring that four of the six combinations were within statistical tolerance. There is a 
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strong likelihood that once the artillery delivery methods in JLink are identical to Janus, 
the ongoing investigations into chemical kills and detection ranges may be resolved. 
After these issues are resolved, this MOP should be analyzed again. One would expect 
the LERs to become more similar for all scenarios once these major factors in the LER 
equation are corrected. 
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VI. CONCLUSIONS AND RECOMMENDATIONS 



This chapter draws conclusions from Chapter V, assessing the current state of 
JLink in terms of varied scenarios, coalition warfare, and specific combat functions. The 
chapter also provides recommendations for further study of JLink issues presented in this 
thesis. 



A. CONCLUSIONS 

1. Varied Scenarios 

Chapter II describes varied scenarios as different types of combat units in the 
offense and defense, negotiating obstacles, under different environmental conditions. 

Such scenarios had yet to be tested in JLink to the extent tested in this thesis. To assess 
whether JLink replicates these scenarios in a way similar to Janus, the MOPs detection 
range, kill range, rounds fired, and LER were analyzed. The first three MOPs allowed for 
analysis of the basic procedures of an engagement, which is critical to arriving at valid 
conclusions about scenario fidelity. The LER offered a quantitative method to assess the 
overall scenario. 

Although JLink did not accurately replicate detection ranges in this experiment, 
the related MOPs of kill range and rounds fired fared well in the overall analysis. Also, 
statistical tests concluded that four of the six JLink LERs were statistically similar to the 
corresponding Janus LERs. These results support the conclusion that JLink is robust 
enough to sufficiently replicate varied scenarios in a manner similar to Janus, despite 
inaccurate inputs in detection ranges. 
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2 . 



Coalition Warfare 



The same MOPs used for drawing conclusions about varied scenarios 
were used to assess JLink’s ability to portray coalition warfare. Notwithstanding the 
detection range inaccuracies, all MOPs analyzed for scenario #2, armored coalition in 
both HL and SWA environments, failed to reject the null hypothesis, except for red kill 
ranges in HL and blue rounds fired in SWA. Given these results, the thesis concludes 
that JLink produces similar results to stand-alone Janus when simulating coalition 
warfare. 



3. Specific Combat Functions 
a. FASCAM 

The MOP used to assess JLink FASCAM fidelity was the number of kills 
resulting from FASCAM minefields. Although the statistical results for FASCAM 
rejected the null hypothesis only once, the trend towards consistently higher Janus 
FASCAM kills indicates that Janus FASCAM may be more lethal than JLink FASCAM. 
Despite the trend, however, JLink appears to provide a fair representation of FASCAM. 
A stronger conclusion must be borne out of additional FASCAM testing which isolates 
the minefield. 



b. Chemical Artillery 

The MOP used to assess JLink chemical artillery was the number of kills 
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generated by chemical artillery rounds. In all cases, the statistical tests rejected the null 
hypothesis that the average number of Janus chemical kills were the same as the average 
number of JLink chemical kills. Given these results, the thesis concludes that JLink’s 
configuration during the experiment did not accurately replicate the effects of Janus 
chemical artillery. 

The investigation into the cause of the disparate chemical data resulted in a 
solution which changed the delivery of JLink chemical artillery from an individual round 
method to a volley method. Changing to the volley method and rerunning the armored 
scenario in SWA produced very similar chemical kills between Janus and JLink. The 
conclusion is that JLink replicates the effects of Janus chemical artillery for an armored 
scenario in SWA terrain when fired in volley. Further, the volley method must be used 
whenever a user simulates a JLink to JLink battle. 

B. RECOMMENDATIONS 

1. Artillery Code Upgrade 

After investigating the cause of the detection range disparities, a brief study was 
conducted without artillery to support the hypothesis that different artillery delivery 
methods contributed to the different detection results. The study, which involved the 
armored scenario in SWA, proved conclusive. Without artillery, the detection ranges 
were statistically similar. The conclusion is that different artillery delivery methods 
produce different detection ranges. 

The recommendation is that JLink follow the volley method used in Janus 
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stand-alone, and that code be rewritten in JLink to fire artillery using the volley method 
when two JLink players are involved in a simulation. 

2. Further Testing of Detection Ranges, Kill Ranges, Rnds Fired, LERs 

Once the JLink code is rewritten using the volley method for artillery, further tests 
are necessary using various scenarios and environments to confirm that JLink scenarios 
are generating similar detection ranges as compared to Janus scenarios. 

Also, the kill ranges for these scenarios should be analyzed to ensure JLink 
scenarios are producing similar kill ranges and LERs as compared to Janus. The 
assumption is that detection ranges significantly affect kill ranges, based on the 
discussion of the engagement sequence. Once the detection issue is resolved, one would 
expect the kill ranges to be more aligned between Janus and JLink. Then, further testing 
should focus on the kill range results of a battalion-size unit attacking in the HL and 
SWA terrain in order to identify the source of interaction between environment and 
mode. 

Rounds fired should also be analyzed in the re-run scenarios. Again, given the 
engagement sequence, one would expect the average rounds fired to be more similar 
between Janus and JLink once detection ranges and kill ranges are corrected. Additional 
testing should focus on units in the defense under both environments to isolate the cause 
of environment and mode interaction for blue rounds fired. 

Since detection ranges, kill ranges, and rounds fired are inputs into the LER 
equation, one would expect the JLink LERs to be more in line with the Janus LERs once 
the detection ranges and kill ranges generate similar results between the two modes. 
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Testing and resolving these issues will lead to a more robust JLink system, capable of 
accurately portraying more diverse scenarios. 

3. Further Testing of FASCAM and Chemical Artillery 

The trend in the data consistently showing greater Janus FASCAM kills requires 
that FASCAM be investigated further. Also, chemical artillery must be tested using 
scenarios other than the previously tested armored scenario in SWA to confirm that the 
JLink code upgrades for chemical rounds sufficiently replicates chemical kills between 
the two modes. Improvements in both FASCAM and chemical artillery will diversify 
JLink and increase its value as a training and analytical tool. 

C. FINAL COMMENTS 

This thesis provides the latest assessment of JLink fidelity. The conclusions and 
recommendations of this thesis will improve distributed interoperability. However, in 
order to achieve the final goal of 100% interoperability between JLink systems, continued 
testing is necessary on the topics mentioned in this thesis and future topics generated by 
the evolving Jlink system. 
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APPENDIX A. RAW DATA 



SCENARIO #1, SWA RESULTS 





FASCAM Kills 


Chem Kills 


LER 


Detection Ranges 


Kill Ranges 


Rounds Fired 


Run# 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JLINK 


Janus 


JLINK 


Janus 


JUNK 


Janus 


JLINK 


1 


4 


4 


4 


0 


1.95 


1.08 


3.400 


3.844 


2.411 


2.636 


236 


248 


2 


2 


2 


6 


0 


2.11 


0.81 


3.351 


3.814 


2.493 


2.638 


119 


235 


3 


0 


1 


5 


0 


2.59 


0.96 


3.278 


3.862 


2.632 


2.673 


247 


201 


4 


6 


0 


4 


0 


2.14 


0.96 


3.294 


3.905 


2.749 


2.876 


183 


163 


5 


2 


3 


4 


0 


2.47 


1.09 


3.298 


3.826 


2.464 


2.671 


255 


191 


6 


1 


2 


4 


1 


1.95 


1.19 


3.394 


3.813 


2.668 


2.641 


150 


302 


7 


3 


2 


4 


0 


2 


1 


3.260 


3.885 


2.391 


2.634 


225 


228 


8 


3 


3 


4 


1 


2.61 


1.41 


3.232 


3.753 


2.739 


3.057 


205 


223 


9 


3 


1 


5 


1 


2.05 


1.39 


3.338 


3.765 


2.561 


2.746 


187 


246 


10 


3 


2 


4 


0 


1.95 


1.72 


3.425 


3.647 


2.650 


2.830 


147 


244 


XBAR 


2.7 


2 


4.4 


0.3 


2.182 


1.161 


3.327 


3.811 


2.576 


2.740 


195.4 


228.1 


VAR 


2.68 


1.33 


0.49 


0.23 


0.072 


0.074 


0.004 


0.006 


0.017 


0.020 


2144.04 


1428.1 


STDEV 


1.64 


1.15 


0.70 


0.48 


0.269 


0.273 


0.065 


0.075 


0.131 


0.141 


46.30 


37.79 



SCENARIO #1, HL RESULTS 





FASCAM Kills 


Chem Kills 


LER 


Detection Ranges 


Kill Ranges 


Rounds Fired 


Run# 


Janus 


JLINK 


Janus 


JLINK 


Janus 


JLINK 


Janus 


JLINK 


Janus 


JLINK 


Janus 


JLINK 


1 


4 


3 


8 


3 


2.09 


1.69 


3.027 


3.544 


2.406 


2.983 


546 


328 


2 


3 


1 


12 


4 


1.46 


1.52 


3.099 


3.772 


2.388 


2.711 


267 


256 


3 


6 


2 


13 


2 


1.36 


1.8 


3.089 


3.733 


2.428 


2.878 


454 


370 


4 


3 


1 


13 


4 


1.29 


1.43 


2.889 


3.581 


2.342 


2.946 


815 


551 


5 


1 


2 


14 


1 


1.95 


1.32 


2.945 


3.601 


2.311 


2.905 


570 


393 


6 


0 


1 


13 


4 


1.56 


1.87 


2.843 


3.455 


2.179 


2.687 


418 


354 


7 


2 


0 


11 


2 


1.37 


1.19 


2.913 


3.680 


2.347 


2.665 


705 


410 


8 


2 


1 


13 


2 


1.6 


1.52 


2.914 


3.305 


2.085 


2.832 


669 


453 


9 


4 


2 


14 


2 


2.5 


1.32 


2.853 


3.699 


2.532 


2.663 


839 


320 


10 


4 


2 


12 


1 


1.41 


1.39 


2.743 


3.552 


2.346 


2.858 


1159 


384 


XBAR 


2.9 


1.5 


12.3 


2.5 


1.659 


1.505 


2.931 


3.592 


2.337 


2.813 


644.2 


381.9 


VAR 


2.99 


0.72 


3.12 


1.39 


0.156 


0.049 


0.013 


0.020 


0.016 


0.015 


64580.18 


6452.77 


STDEV 


1.73 


0.85 


1.77 


1.18 


0.395 


0.221 


0.112 


0.140 


0.126 


0.121 


254.13 


80.33 



SCENARIO #2, SWA RESULTS 





FASCAM Kills 


Chem Kills 


LER 


Detection Ranges 


Kill Ranges 


Rounds Fired 


Run # 


Janus 


JLINK 


Janus 


JLINK 


Janus 


JLINK 


Janus 


JLINK 


Janus 


JLINK 


Janus 


JLINK 


1 


2 


3 


4 


0 


1.26 


1.4 


3.196 


3.364 


2.574 


2.635 


141 


147 


2 


1 


0 


3 


0 


1.13 


0.75 


3.230 


3.415 


2.360 


2.640 


136 


245 


3 


4 


1 


5 


0 


1.52 


1.17 


3.069 


3.511 


2.746 


2.528 


215 


189 


4 


4 


2 


4 


1 


1.67 


1.21 


2.955 


3.263 


2.447 


2.563 


140 


231 


5 


2 


3 


3 


0 


1.38 


1.65 


3.018 


3.379 


2.761 


2.735 


151 


227 


6 


4 


1 


3 


0 


1.48 


1.05 


3.149 


3.234 


2.499 


2.521 


387 


152 


7 


4 


0 


4 


0 


1.48 


1.22 


3.086 


3.297 


2.462 


2.527 


182 


228 


8 


2 


3 


4 


0 


1.29 


1.33 


3.157 


3.395 


2.358 


2.484 


150 


184 


9 


0 


2 


4 


0 


1.26 


1.15 


3.217 


3.366 


2.553 


2.960 


304 


180 


10 


0 


0 


4 


0 


1.3 


0.81 


3.135 


3.200 


2.469 


3.262 


199 


227 


XBAR 


2.3 


1.5 


3.8 


0.1 


1.377 


1.174 


3.121 


3.342 


2.523 


2.685 


200.5 


201 


VAR 


2.68 


1.61 


0.4 


0.1 


0.025 


0.070 


0.008 


0.009 


0.020 


0.061 


6934.5 


1234.22 


STDEV 


1.64 


1.27 


0.63 


0.32 


0.160 


0.265 


0.089 


0.094 


0.140 


0.247 


83.27 


35.13 
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SCENARIO #2, HL RESULTS 





FASCAM Kills 


Chem Kills 


LER 


Detection Ranges 


Kill Ranges 


Rounds Fired 


Run # 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


1 


1 


0 


11 


2 


1.3 


2.4 


2.106 


2.398 


2.247 


2.745 


385 


353 


2 


4 


1 


12 


3 


1.07 


1.64 


2.054 


2.594 


2.414 


2.715 


516 


362 


3 


2 


1 


9 


2 


1.48 


2.1 


2.115 


2.370 


2.294 


2.677 


607 


452 


4 


6 


0 


14 


2 


1.58 


2 


2.105 


2.515 


2.281 


2.639 


448 


312 


5 


2 


0 


14 


5 


1.78 


1.67 


2.150 


2.519 


2.463 


2.825 


478 


520 


6 


2 


1 


14 


2 


1.46 


1.92 


2.085 


2.496 


2.182 


2.688 


591 


711 


7 


2 


1 


12 


3 


1.71 


1.5 


2.084 


2.322 


2.167 


2.881 


795 


344 


8 


2 


0 


13 


4 


1.5 


1.18 


2.081 


2.400 


2.115 


2.778 


813 


688 


9 


4 


1 


13 


3 


1.44 


2.1 


2.140 


2.401 


2.164 


2.609 


711 


522 


10 


1 


1 


11 


3 


1.57 


1.84 


2.105 


2.572 


2.489 


2.522 


604 


591 


XBAR 


2.6 


0.6 


12.3 


2.9 


1.489 


1.835 


2.103 


2.459 


2.282 


2.708 


594.8 


485.5 


VAR 


2.49 


0.27 


2.68 


0.99 


0.040 


0.122 


0.001 


0.008 


0.018 


0.011 


20762.18 


21062.72 


STDEV 


1.58 


0.52 


1.64 


0.99 


0.201 


0.350 


0.028 


0.092 


0.133 


0.106 


144.09 


145.13 



SCENARIO #3, SWA RESULTS 





FASCAM Kills 


Chem Kills 


LER 


Detection Ranges 


Kill Ranges 


Rounds Fired 


Run # 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


1 


2 


0 


9 


0 


1.28 


0.94 


1.935 


2.036 


1.334 


1.526 


2640 


2396 


2 


0 


2 


15 


0 


1.36 


0.88 


1.932 


2.028 


1.437 


1.528 


2486 


2253 


3 


0 


0 


12 


0 


1.53 


0.79 


1.956 


2.050 


1.378 


1.493 


2417 


2202 


4 


1 


0 


6 


0 


2.28 


0.95 


1.895 


2.060 


1.309 


1.531 


2890 


2309 


5 


0 


0 


9 


3 


1.92 


1.06 


1.937 


2.011 


1.312 


1.449 


2712 


2163 


6 


0 


0 


12 


3 


1.39 


0.77 


1.964 


2.055 


1.361 


1.560 


2553 


2111 


7 


5 


0 


12 


0 


2.47 


0.95 


1.906 


2.026 


1.408 


1.468 


2981 


2292 


8 


0 


0 


6 


0 


1.17 


0.92 


1.931 


2.050 


1.436 


1.492 


2339 


2156 


9 


2 


0 


6 


0 


0.92 


1.03 


1.963 


2.057 


1.441 


1.484 


2288 


2207 


10 


1 


0 


12 


0 


1.25 


1.08 


1.988 


2.032 


1.371 


1.485 


2650 


2314 


XBAR 


1.1 


0.2 


9.9 


0.6 


1.557 


0.937 


1.941 


2.041 


1.379 


1.501 


2595.6 


2240.3 


VAR 


2.54 


0.4 


10.1 


1.6 


0.254 


0.011 


0.001 


0.000 


0.003 


0.001 


51096.71 


7733.79 


STDEV 


1.60 


0.63 


3.18 


1.26 


0.504 


0.104 


0.028 


0.016 


0.051 


0.034 


226.05 


87.94 



SCENARIO #3* HL RESULTS 





FASCAM Kills 


Chem Kills 


LER 


Detection Ranges 


Kill Ranges 


Rounds Fired 


Run # 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


1 


2 


1 


69 


33 


5.63 


2.22 


1.588 


1.713 


1.323 


1.443 


2815 


2896 


2 


0 


1 


69 


33 


2.71 


2.48 


1.610 


1.686 


1.294 


1.396 


2383 


2713 


3 


0 


1 


69 


48 


2.72 


2.49 


1.630 


1.677 


1.303 


1.374 


2895 


3310 


4 


1 


0 


69 


42 


3.74 


3.65 


1.649 


1.658 


1.328 


1.306 


3308 


2921 


5 


2 


1 


72 


33 


3.51 


2.06 


1.579 


1.716 


1.228 


1.495 


2412 


3121 


6 


0 


1 


69 


21 


2.18 


2.6 


1.665 


1.667 


1.372 


1.369 


2786 


2823 


7 


0 


0 


72 


36 


2.59 


1.32 


1.590 


1.770 


1.382 


1.511 


2284 


2978 


8 


1 


1 


69 


30 


2.42 


2.35 


1.605 


1.725 


1.250 


1.416 


2816 


3246 


9 


3 


0 


69 


39 


2.93 


2.9 


1.589 


1.664 


1.376 


1.345 


2727 


2565 


10 


1 


0 


69 


51 


2.69 


1.53 


1.593 


1.733 


1.271 


1.567 


2705 


2782 


XBAR 


1 


0.6 


69.6 


36.6 


3.112 


2.36 


1.610 


1.701 


1.313 


1.422 


2713.1 


2935.5 


VAR 


1.11 


0.27 


1.6 


77.6 


1.004 


0.436 


0.001 


0.001 


0.003 


0.007 


88588.1 


55293.61 


STDEV 


1.05 


0.52 


1.26 


8.81 


1.002 


0.660 


0.029 


0.037 


0.054 


0.082 


297.64 


235.15 
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APPENDIX B. RAW DATA SEPARATED BY SIDE 



SCENARIO #1, SWA RESULTS 





Det Ranges Blue 


Det Ranges Red 


Kill Ranges Blue 


Kill Ranges Red 


Rnds Fired Blue 


Rnds Fired Red 


Run# 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


1 


3.847 


4.231 


2.381 


3.398 


1.986 


2.356 


2.704 


2.987 


60 


27 


176 


221 


2 


3.707 


4.139 


2.344 


3.393 


1.903 


2.413 


2.805 


3.028 


54 


23 


65 


212 


3 


3.616 


4.220 


2.412 


3.510 


1.952 


2.343 


2.901 


3.064 


172 


29 


75 


172 


4 


3.656 


4.274 


2.480 


3.425 


2.020 


2.393 


3.153 


3.404 


87 


29 


96 


134 


5 


3.570 


4.164 


2.661 


3.452 


1.862 


2.559 


2.725 


2.826 


102 


23 


153 


168 


6 


3.825 


4.068 


2.308 


3.538 


1.905 


2.353 


3.083 


2.930 


63 


55 


87 


247 


7 


3.570 


4.245 


2.593 


3.502 


1.929 


2.381 


2.661 


2.909 


108 


30 


117 


198 


8 


3.651 


4.187 


2.308 


3.108 


3.049 


2.458 


2.613 


3.630 


85 


30 


120 


193 


9 


3.764 


4.087 


2.414 


3.418 


2.039 


2.404 


2.860 


3.037 


64 


38 


123 


208 


10 


3.684 


4.094 


2.800 


3.148 


2.096 


2.276 


2.968 


3.185 


58 


44 


89 


200 


XBAR 


3.689 


4.171 


2.470 


3.389 


2.074 


2.394 


2.847 


3.100 


85.3 


32.8 


110.1 


195.3 


VAR 


0.010 


0.005 


0.027 


0.021 


0.123 


0.006 


0.033 


0.061 


1292.23 


101.73 


1215.43 


983.79 


STDEV 


0.098 


0.072 


0.165 


0.146 


0.350 


0.076 


0.181 


0.246 


35.95 


10.09 


34.86 


31.37 



SCENARIO #1, HL RESULTS 





Det Ranges Blue 


Det Ranges Red 


Kill Ranges Blue 


Kill Ranges Red 


Rnds Fired Blue 


Rnds Fired Red 


Run # 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


1 


3.389 


3.845 


2.022 


2.959 


2.000 


2.790 


2.629 


3.112 


171 


65 


375 


263 


2 


3.419 


4.047 


2.076 


3.078 


1.821 


2.302 


2.835 


3.003 


87 


59 


180 


197 


3 


3.419 


4.060 


2.068 


3.092 


2.249 


2.580 


2.590 


3.060 


150 


65 


304 


305 


4 


3.259 


3.970 


2.142 


2.886 


2.138 


3.019 


2.534 


2.889 


78 


59 


737 


492 


5 


3.362 


3.901 


1.880 


3.086 


1.920 


2.689 


2.534 


3.078 


149 


60 


421 


333 


6 


3.211 


3.913 


2.070 


2.530 


1.726 


2.293 


2.470 


2.908 


147 


62 


271 


292 


7 


3.258 


4.060 


2.103 


2.797 


1.851 


2.427 


2.740 


2.866 


86 


49 


619 


361 


8 


3.349 


3.765 


1.892 


2.686 


1.617 


2.601 


2.410 


3.000 


87 


62 


582 


391 


9 


3.338 


4.071 


1.849 


2.835 


2.248 


2.237 


2.664 


3.014 


90 


53 


749 


267 


10 


3.296 


3.873 


1.903 


2.871 


1.978 


2.623 


2.670 


3.045 


78 


47 


1081 


337 


XBAR 


3.330 


3.951 


2.000 


2.882 


1.955 


2.556 


2.607 


2.997 


112.3 


58.1 


531.9 


323.8 


VAR 


0.005 


0.012 


0.012 


0.034 


0.045 


0.061 


0.016 


0.007 


1360.01 


40.32 


75729.21 


6535.07 


STDEV 


0.072 


0.107 


0.108 


0.183 


0.212 


0.246 


0.127 


0.084 


36.88 


6.35 


275.19 


80.84 



SCENARIO #2, SWA RESULTS 





Det Ranges Blue 


Det Ranges Red 


Kill Ranges Blue 


Kill Ranges Red 


Rnds Fired Blue 


Rnds Fired Red 


Run# 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


1 


3.841 


4.128 


2.316 


3.271 


1.871 


2.098 


3.277 


3.115 


59 


35 


82 


112 


2 


3.718 


4.118 


2.218 


3.365 


1.788 


2.205 


3.019 


3.026 


34 


74 


102 


171 


3 


3.685 


3.992 


2.292 


3.325 


2.687 


2.098 


2.796 


2.958 


42 


29 


173 


160 


4 


3.614 


4.075 


2.219 


3.166 


1.742 


2.197 


2.993 


2.902 


49 


32 


91 


199 


5 


3.584 


4.151 


2.115 


3.376 


2.672 


2.179 


2.840 


3.162 


72 


38 


79 


189 


6 


3.173 


4.090 


2.030 


2.998 


1.766 


2.246 


3.173 


2.838 


52 


28 


335 


124 


7 


3.784 


4.103 


2.328 


3.219 


1.864 


2.121 


2.929 


2.901 


74 


36 


108 


192 


8 


3.611 


4.090 


2.091 


3.234 


1.771 


2.110 


2.900 


2.843 


62 


30 


88 


154 


9 


3.558 


4.176 


2.208 


3.492 


1.790 


2.375 


3.179 


3.609 


70 


22 


234 


158 


10 


3.625 


3.812 


2.302 


2.977 


1.916 


3.626 


2.940 


2.684 


51 


29 


148 


198 


XBAR 


3.619 


4.073 


2.212 


3.242 


1.987 


2.325 


3.005 


3.004 


56.5 


35.3 


144 


165.7 


VAR 


0.033 


0.011 


0.011 


0.027 


0.136 


0.216 


0.025 


0.065 


176.5 


206.01 


6945.78 


918.46 


STDEV 


0.181 


0.104 


0.103 


0.163 


0.369 


0.465 


0.158 


0.255 


13.29 


14.35 


83.34 


30.31 
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SCENARIO #2, HL RESULTS 





Det Ranges Blue 


Det Ranges Red 


Kill Ranges Blue 


Kill Ranges Red 


Rnds Fired Blue 


Rnds Fired Red 


Run # 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


1 


3.312 


3.900 


1.944 


2.913 


2.033 


1.738 


2.423 


3.164 


78 


91 


307 


262 


2 


3.573 


4.036 


1.907 


3.021 


2.073 


2.226 


2.797 


3.028 


41 


60 


475 


302 


3 


3.361 


3.991 


1.792 


2.577 


2.011 


1.741 


2.500 


3.145 


64 


59 


543 


393 


4 


3.301 


3.982 


1.866 


2.648 


1.683 


2.066 


2.737 


2.932 


83 


67 


367 


245 


5 


3.465 


3.904 


1.899 


2.709 


1.984 


2.599 


2.769 


2.963 


56 


81 


422 


439 


6 


3.245 


3.890 


2.025 


2.496 


1.718 


2.275 


2.541 


2.908 


67 


123 


524 


588 


7 


3.401 


4.074 


1.807 


2.474 


1.536 


2.544 


2.599 


3.112 


66 


56 


729 


28 8 


8 


3.338 


3.970 


1.839 


2.717 


1.612 


2.561 


2.492 


2.967 


116 


58 


697 


630 


9 


3.335 


3.838 


1.993 


2.227 


1.582 


1.887 


2.633 


2.970 


63 


72 


648 


450 


10 


3.286 


3.895 


1.806 


2.463 


2.083 


1.809 


2.765 


2.927 


82 


69 


522 


522 


XBAR 


3.362 


3.948 


1.888 


2.624 


1.831 


2.145 


2.626 


3.012 


71.6 


73.6 


523.4 


411.9 


VAR 


0.009 


0.006 


0.007 


0.054 


0.050 


0.120 


0.018 


0.009 


401.6 


424.04 


19137.16 


18999.88 


STDEV 


0.096 


0.074 


0.081 


0.232 


0.224 


0.346 


0.135 


0.095 


20.04 


20.59 


138.34 


137.84 



SCENARIO #3, SWA RESULTS 





Det Ranges Blue 


Det Ranges Red 


Kill Ranges Blue 


Kill Ranges Red 


Rnds Fired Blue 


Rnds Fired Red 


Run # 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


1 


2.050 


2.038 


1.811 


2.034 


1.359 


1.616 


1.313 


1.427 


875 


646 


1765 


1750 


2 


2.071 


2.070 


1.793 


1.990 


1.481 


1.611 


1.405 


1.433 


866 


628 


1620 


1625 


3 


2.057 


2.116 


1.821 


1.992 


1.596 


1.594 


1.235 


1.365 


972 


557 


1445 


1645 


4 


2.014 


2.068 


1.747 


2.052 


1.372 


1.683 


1.282 


1.370 


1295 


654 


1595 


1655 


5 


2.046 


2.034 


1.778 


1.989 


1.450 


1.635 


1.240 


1.271 


1137 


638 


1575 


1525 


6 


2.087 


2.107 


1.833 


2.007 


1.525 


1.609 


1.244 


1.495 


833 


546 


1720 


1565 


7 


2.027 


2.079 


1.746 


1.973 


1.510 


1.610 


1.366 


1.315 


1456 


617 


1525 


1675 


8 


2.058 


2.075 


1.785 


2.028 


1.540 


1.631 


1.345 


1.338 


749 


601 


1590 


1555 


9 


2.126 


2.066 


1.779 


2.048 


1.462 


1.659 


1.416 


1.313 


658 


637 


1630 


1570 


10 


2.138 


2.097 


1.817 


1.971 


1.476 


1.611 


1.286 


1.365 


855 


689 


1795 


1625 


XBAR 


2.067 


2.075 


1.791 


2.008 


1.477 


1.626 


1.313 


1.369 


969.6 


621.3 


1626 


1619 


VAR 


0.002 


0.001 


0.001 


0.001 


0.005 


0.001 


0.005 


0.004 


63081.38 


1896.46 


11621.11 


4510 


STDEV 


0.040 


0.027 


0.030 


0.030 


0.072 


0.027 


0.067 


0.067 


251.16 


43.55 


107.80 


67.16 



SCENARIO #3, HL RESULTS 





Det Ranges Blue 


Det Ranges Red 


Kill Ranges Blue 


Kill Ranges Red 


Rnds Fired Blue 


Rnds Fired Red 


Run # 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


1 


1.613 


1.718 


1.547 


1.702 


1.327 


1.377 


1.322 


1.474 


1960 


1276 


855 


1620 


2 


1.641 


1.707 


1.572 


1.652 


1.312 


1.359 


1.287 


1.412 


1068 


1303 


1315 


1410 


3 


1.665 


1.703 


1.586 


1.634 


1.312 


1.363 


1.300 


1.378 


1345 


1535 


1550 


1775 


4 


1.675 


1.676 


1.612 


1.626 


1.193 


1.420 


1.364 


1.275 


1833 


1636 


1475 


1285 


5 


1.614 


1.733 


1.536 


1.690 


1.319 


1.336 


1.202 


1.572 


1257 


1361 


1155 


1760 


6 


1.723 


1.677 


1.579 


1.651 


1.323 


1.376 


1.394 


1.366 


1151 


1268 


1635 


1555 


7 


1.632 


1.786 


1.538 


1.744 


1.399 


1.373 


1.376 


1.617 


1134 


1018 


1150 


1960 


8 


1.644 


1.747 


1.547 


1.688 


1.241 


1.362 


1.253 


1.438 


1131 


1486 


1685 


1760 


9 


1.623 


1.690 


1.543 


1.620 


1.299 


1.398 


1.403 


1.327 


1352 


1330 


1375 


1235 


10 


1.646 


1.753 


1.530 


1.709 


1.286 


1.386 


1.265 


1.687 


1245 


1202 


1460 


1580 


XBAR 


1.648 


1.719 


1.559 


1.672 


1.301 


1.375 


1.317 


1.455 


1347.6 


1341.5 


1365.5 


1594 


VAR 


0.001 


0.001 


0.001 


0.002 


0.003 


0.001 


0.004 


0.018 


93197.38 


31205.83 


64852.5 


53604.44 


STDEV 


0.033 


0.036 


0.027 


0.041 


0.055 


0.023 


0.067 


0.133 


305.28 


176.65 


254.66 


231.53 
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APPENDIX C. RAW DATA FOR SAME SEED JLINK RUNS 



SCENARIO #1, SWA JLINK RESULTS 



Seed# 


Run# 


FASCAM Kills 


Chem Kills 


LER 


Detection Ranges 


Kill Ranges 


Rounds Fired 


55 


1 


4 


0 


1.08 


3.844 


2.636 


248 


55 


2 


1 


1 


1 


3.816 


2.707 


260 


55 


3 


3 


0 


1.14 


3.851 


2.774 


158 


55 


4 


1 


0 


2.09 


3.705 


2.804 


252 


55 


5 


2 


0 


1.13 


3.853 


2.851 


159 


55 


6 


1 


0 


1.3 


3.786 


2.635 


182 


55 


7 


0 


0 


1.17 


3.809 


2.711 


257 


55 


8 


2 


0 


0.88 


3.822 


2.645 


196 


55 


9 


1 


0 


0.92 


3.837 


2.604 


224 


55 


10 


4 


0 


1.17 


3.956 


2.700 


230 




XBAR 


1.9 


0.1 


1.188 


3.828 


2.707 


216.6 




VAR 


1.88 


0.1 


0.116 


0.004 


0.007 


1595.82 




STDEV 


1.37 


0.32 


0.341 


0.063 


0.081 


39.95 



SCENARIO #1, HL JLINK RESULTS 



Seed# 


Run # 


FASCAM Kills 


Chem Kills 


LER 


Detection Ranges 


Kill Ranges 


Rounds Fired 


55 


1 


3 


3 


1.69 


3.544 


2.983 


328 


55 


2 


1 


1 


1.21 


3.531 


2.757 


377 


55 


3 


2 


3 


1.32 


3.583 


2.776 


248 


55 


4 


2 


2 


1.56 


3.624 


2.983 


219 


55 


5 


0 


2 


1.43 


3.787 


3.120 


306 


55 


6 


2 


2 


1.1 


3.633 


2.763 


466 


55 


7 


1 


4 


1.46 


3.438 


2.805 


307 


55 


8 


2 


4 


1.91 


3.583 


2.811 


302 


55 


9 


1 


1 


1.15 


3.689 


2.617 


212 


55 


10 


2 


3 


1.92 


3.691 


2.811 


277 




XBAR 


1.6 


2.5 


1.475 


3.610 


2.843 


304.2 




VAR 


0.71 


1.17 


0.087 


0.010 


0.021 


5746.62 




STDEV 


0.84 


1.08 


0.295 


0.098 


0.145 


75.81 



SCENARIO #2, SWA JLINK RESULTS 



Seed# 


Run # 


FASCAM Kills 


Chem Kills 


LER 


Detection Ranges 


Kill Ranges 


Rounds Fired 


55 


1 


3 


0 


1.4 


3.364 


2.635 


147 


55 


2 


0 


0 


1.17 


3.369 


2.859 


106 


55 


3 


0 


0 


1.19 


3.360 


2.441 


176 


55 


4 


2 


1 


1.2 


3.514 


2.587 


158 


55 


5 


2 


0 


1.35 


3.428 


2.550 


204 


55 


6 


2 


0 


1.26 


3.451 


3.480 


121 


55 


7 


0 


0 


1.32 


3.223 


2.500 


258 


55 


8 


1 


1 


1.17 


3.283 


2.831 


243 


55 


9 


1 


0 


0.96 


3.324 


2.539 


256 


55 


10 


0 


1 


1.31 


3.339 


2.467 


319 




XBAR 


1.1 


0.3 


1.233 


3.365 


2.689 


198.8 




VAR 


1.21 


0.23 


0.016 


0.007 


0.097 


4748.62 




STDEV 


1.10 


0.48 


0.125 


0.084 


0.312 


68.91 
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SCENARIO #2, HL JLINK RESULTS 



Seed# 


Run# 


FASCAM Kills 


Chem Kills 


LER 


Detection Ranges 


Kill Ranges 


Rounds Fired 


55 


1 


0 


2 


2.4 


2.398 


2.745 


353 


55 


2 


2 


1 


1.36 


2.254 


2.501 


527 


55 


3 


1 


1 


1.5 


2.189 


2.619 


362 


55 


4 


2 


2 


1.63 


2.469 


2.692 


319 


55 


5 


2 


0 


1.31 


2.264 


2.687 


426 


55 


6 


2 


3 


1.77 


2.435 


2.792 


450 


55 


7 


2 


4 


2.63 


2.486 


2.787 


461 


55 


8 


0 


1 


1.54 


2.518 


2.712 


336 


55 


9 


2 


2 


1.59 


2.284 


2.708 


531 


55 


10 


3 


2 


1.79 


2.415 


2.498 


549 




XBAR 


1.6 


1.8 


1.752 


2.371 


2.674 


431.4 




VAR 


0.93 


1.29 


0.188 


0.013 


0.011 


7426.49 




STDEV 


0.97 


1.14 


0.433 


0.114 


0.105 


86.18 



SCENARIO #3, SWA JLINK RESULTS 



Seed # 


Run # 


FASCAM Kills 


Chem Kills 


LER 


Detection Ranges 


Kill Ranges 


Rounds Fired 


55 


1 


0 


0 


0.94 


2.036 


1.526 


2396 


55 


2 


0 


0 


1.18 


2.030 


1.423 


2268 


55 


3 


0 


0 


0.87 


2.059 


1.508 


2126 


55 


4 


0 


0 


0.78 


2.053 


1.511 


2228 


55 


5 


0 


3 


1.03 


2.057 


1.413 


2212 


55 


6 


1 


0 


1.09 


2.060 


1.441 


2285 


55 


7 


0 


0 


0.95 


2.055 


1.482 


2340 


55 


8 


0 


0 


0.95 


2.088 


1.564 


2321 


55 


9 


0 


0 


0.95 


2.022 


1.520 


2215 


55 


10 


0 


0 


1 


2.046 


1.488 


2215 




XBAR 


0.1 


0.3 


0.974 


2.051 


1.488 


2260.6 




VAR 


0.1 


0.9 


0.012 


0.000 


0.002 


6070.71 




STDEV 


0.32 


0.95 


0.111 


0.019 


0.049 


77.91 



SCENARIO #3, HL JLINK RESULTS 



Seed # 


Run# 


FASCAM Kills 


Chem Kills 


LER 


Detection Ranges 


Kill Ranges 


Rounds Fired 


55 


1 


1 


33 


2.22 


1.713 


1.443 


2896 


55 


2 


1 


36 


2.29 


1.726 


1.481 


3137 


55 


3 


2 


42 


2.8 


1.735 


1.414 


3455 


55 


4 


0 


48 


2.81 


1.724 


1.420 


3066 


55 


5 


0 


33 


1.52 


1.738 


1.548 


2592 


55 


6 


0 


24 


1.58 


1.799 


1.493 


2835 


55 


7 


0 


36 


1.93 


1.696 


1.377 


2872 


55 


8 


2 


30 


2.19 


1.686 


1.322 


3085 


55 


9 


0 


33 


1.95 


1.692 


1.402 


2839 


55 


10 


0 


21 


1.87 


1.735 


1.406 


2798 




XBAR 


0.6 


33.6 


2.116 


1.724 


1.431 


2957.5 




VAR 


0.71 


61.6 


0.196 


0.001 


0.004 


56325.17 




STDEV 


0.84 


7.85 


0.442 


0.032 


0.064 


237.33 
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APPENDEX D. POWER DATA 



Scenario #1, SWA 




A 




<i> 


Power 


FASCAM 


1 


0.070 


22.54 


1.00 


Chemical Kills 


2 


0.601 


5.26 


1.00 


LER 


.3 


0.271 


1.75 


.72 


Detection Range 


.3 


0.070 


6.76 


1.00 


Kill Range 


.2 


0.136 


2.32 


.88 


Rounds Fired 


40 


42.262 


1.50 


.54 



Scenario #1, HL 




A 






Power 


FASCAM 


1 


1.362 


1.16 


.31 


Chemical Kills 


2 


1.502 


2.11 


.81 


LER 


.3 


0.320 


1.48 


.54 


Detection Range 


.3 


0.127 


3.73 


1.00 


Kill Range 


.2 


0.124 


2.56 


.93 


Rounds Fired 


130 


188.458 


1.09 


.29 



Scenario #2, SWA 




A 




♦ 


Power 


FASCAM 


1 


1.464 


1.08 


.28 


Chemical Kills 


2 


0.5 


6.33 


1.00 


LER 


.3 


0.219 


2.17 


.83 


Detection Range 


.3 


0.091 


5.19 


1.00 


Kill Range 


.2 


0.200 


1.58 


.56 


Rounds Fired 


40 


63.909 


0.99 


.25 
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Scenario #2, HL 




A 




4> 


Power 


FASCAM 


1 


1.174 


1.35 


.40 


Chemical Kills 


2 


1.354 


2.34 


.88 


LER 


.3 


0.285 


1.66 


.62 


Detection Range 


.3 


0.0681 


6.96 


1.00 


Kill Range 


.2 


0.120 


2.63 


.94 


Rounds Fired 


120 


144.611 


1.31 


.40 



Scenario #3, SWA 




A 




<t> 


Power 


FASCAM 


1 


1.213 


1.30 


.40 


Chemical Kills 


20 


2.419 


13.07 


1.00 


LER 


.3 


0.364 


1.30 


.40 


Detection Range 


.2 


0.0228 


13.90 


1.00 


Kill Range 


.05 


0.043 


1.84 


.69 


Rounds Fired 


520 


171.509 


4.79 


1.00 



Scenario #3, HL 




A 




4> 


Power 


FASCAM 


1 


0.830 


1.91 


.72 


Chemical Kills 


20 


6.293 


5.03 


1.00 


LER 


.3 


0.849 


0.56 


<.20 


Detection Range 


.2 


0.033 


9.60 


1.00 


Kill Range 


.05 


0.069 


1.14 


.31 


Rounds Fired 


540 


268.218 


3.18 


.986 
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APPENDIX E. ANALYSIS OF VARIANCE (ANOVA) RESULTS 



ANOVA FOR FASCAM KILLS 

Analysis of Variance Table 
Response : MOE 



Terms added sequentially (first to last) 





Df 


Sum of Sq 


Mean Sq 


F Value 


Seen 


2 


49.7167 


24.85833 


15.62689 


Env 


1 


0.3000 


0.30000 


0 . 18859 


Mode 


1 


32.0333 


32.03333 


20.13737 


Seen : Env 


2 


1.0500 


0.52500 


0.33003 


Seen : Mode 


2 


2.8167 


1.40833 


0.88533 


Env : Mode 


1 


1.6333 


1.63333 


1.02678 


Seen : Env : Mode 


2 


3.8167 


1.90833 


1.19965 


Residuals 


108 


171.8000 


1.59074 





Pr (F) 
0.0000011 
0.6649580 
0.0000181 
0.7196211 
0.4155499 
0.3131820 
0.3052820 



Interaction plots for scenario renvironment, scenario:mode, and environmentrmode. 
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ANOVA FOR CHEMICAL KILLS 



Analysis of Variance Table 
Response: MOE 



Terms added sequentially (first to last) 



Seen 

Env 

Mode 

Seen : Env 

Seen : Mode 

Env : Mode 

Seen : Env .-Mode 

Residuals 



Df 

2 

1 

1 

2 

2 

1 

2 

108 



Sum of Sq 

15811.47 
11427.01 

4002 . 07 

12043.47 
1383 .20 
1026 . 68 

540.00 

902.70 



Mean Sq 

7905.73 
11427 .01 

4002 . 07 

6021.73 
691.60 

1026 . 68 
270.00 
8.36 



F Value 
945.850 
1367.140 
478 . 813 
720.447 
82.744 
122.833 
32.303 



Pr (F) 
0 . 000000e+000 
0 . 000000e+000 
0 . 000000e+000 
0 .000000e+000 
0 . 000000e+000 
0 . 000000e+000 
1 ,008882e-011 



Interaction plots for scenario:environment, scenario:mode, and environment:mode. 
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ANOVA FOR LERs 



Analysis of Variance Table 
Response: MOE 



Terms added sequentially (first to last) 





Df 


Sum of Sq 


Mean Sq 


F Value 


Seen 


2 


5.75032 


2.87516 


14.90381 


Env 


1 


10.63265 


10.63265 


55 . 11590 


Mode 


1 


4 . 81601 


4.81601 


24.96450 


Seen : Env 


2 


13.11248 


6.55624 


33 . 98523 


Seen :Mode 


2 


3.39263 


1.69632 


8.79310 


Env : Mode 


1 


1.37388 


1.37388 


7.12171 


Seen : Env : Mode 


2 


1.30240 


0.65120 


3.37560 


Residuals 


108 


20.83476 


0.19291 





Pr (F) 

0.00000192 
0.00000000 
0.00000226 
0.00000000 
0.00028975 
0 . 00878978 
0.03784347 



Interaction plots for scenario:environment, scenario:mode, and environment:mode. 
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ANOVA for Detection Range Blue 



Analysis of Variance Table 
Response: MOE 



Terms added sequentially (first to last) 



Seen 

Env 

Mode 

Seen : Env 

Seen -.Mode 

Env : Mode 

Seen : Env : Mode 

Residuals 



Df Sum of Sq 
2 95.34164 

1 2.51797 

1 4.11438 

2 0.19275 

2 1.64627 

1 0.09369 

2 0.00861 

108 0.85094 



Mean Sq F V 
47.67082 6050 
2.51797 319 

4.11438 522 

0.09638 12 

0.82313 104 

0.09369 11 

0.00431 0 

0.00788 



lue Pr (F) 
279 0.0000000 
576 0.0000000 
188 0.0000000 
232 0.0000163 
470 0.0000000 
892 0.0008050 
547 0.5805110 



Interaction plots for scenario:environment, scenario: mode, and environment: mode. 




Scenario - Mode Interaction 



2 3 

Scenario # 




Jlink 

Janus 



Scenario # 



Environment - Mode Interaction 




Jlink 

Janus 
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ANOVA for Detection Range Red 



Analysis of Variance Table 
Response : MOE 



Terms added sequentially (first to last) 





Df 


Sum of Sq 


Mean Sq 


F Value 


Pr (F) 


Seen 


2 


19.16607 


9.58304 


588.7363 


0 . 00000000 


Env 


1 


5.15815 


5.15815 


316.8920 


0.00000000 


Mode 


1 


12.66121 


12 . 66121 


777 . 8444 


0.00000000 


Seen : Env 


2 


0.25595 


0.12797 


7 . 8622 


0 . 00064909 


Seen : Mode 


2 


3.52387 


1.76194 


108.2451 


0 . 00000000 


Env : Mode 


1 


0.15869 


0.15869 


9.7494 


0 . 00230238 


Seen : Env : Mode 


2 


0.08810 


0.04405 


2.7063 


0.07131454 


Residuals 


108 


1.75795 


0.01628 







Interaction plots for scenario:environment, scenario:mode, and environment:mode. 
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ANOVA for Kill Range Blue 



Analysis of Variance Table 
Response: MOE 



Terms added sequentially (first to last) 





Df 


Sum of Sq 


Mean Sq 


F Value 


Seen 


2 


14 . 17509 


7 . 087543 


111.0971 


Env 


1 


0.43093 


0.430931 


6.7548 


Mode 


1 


2.68719 


2 . 687186 


42.1216 


Seen : Env 


2 


0.31075 


0 . 155376 


2.4355 


Seen : Mode 


2 


0 . 62021 


0.310106 


4.8609 


Env : Mode 


1 


0.02739 


0 . 027389 


0.4293 


Seen : Env : Mode 


2 


0 . 18702 


0 . 093509 


1.4658 


Residuals 


108 


6.88996 


0.063796 





Pr (F) 
0.0000000 
0.0106554 
0 . 0000000 
0.0923476 
0.0095194 
0.5137113 
0.2354604 



Interaction plots for scenario:environment, scenario:mode, and environment:mode. 




Scenario # Scenario * 
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ANOVA for Kill Range Red 



Analysis of Variance Table 
Response: MOE 



Terms added sequentially (first to last) 





Df 


Sum of Sq 


Mean Sq 


F Value 


Seen 


2 


62 . 94847 


31.47423 


1426.979 


Env 


1 


0.32467 


0.32467 


14 . 720 


Mode 


1 


1 . 24480 


1.24480 


56.437 


Seen : Env 


2 


0.33175 


0 . 16587 


7.520 


Seen : Mode 


2 


0 .25323 


0 . 12661 


5.740 


Env : Mode 


1 


0.30617 


0.30617 


13 . 881 


Seen : Env : Mode 


2 


0.13178 


0 . 06589 


2 . 987 


Residuals 


108 


2.38211 


0 . 02206 





Pr (F) 

0.00000000 
0.00021033 
0.00000000 
0.00087544 
0.00427329 
0.00031162 
0 . 05460434 



Interaction plots for scenario: environment, scenario:mode, and environment:mode. 




Scenario # Scenario # 
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ANOVA for Rounds Fired Blue 



Analysis of Variance Table 
Response: MOE 



Terms added sequentially (first to last) 





Df 


Sum of Sq 


Mean Sq 


F Value 


Seen 


2 


26900478 


13450239 


834.6258 


Env 


1 


1207813 


1207813 


74.9482 


Mode 


1 


192240 


192240 


11.9290 


Seen : Env 


2 


1821263 


910631 


56.5073 


Seen : Mode 


2 


151142 


75571 


4.6894 


Env : Mode 


1 


110231 


110231 


6.8402 


Seen: Env: Mode 


2 


183874 


91937 


5.7049 


Residuals 


108 


1740452 


16115 





Pr (F) 

0.00000000 

0.00000000 

0.00079057 

0.00000000 

0.01114398 

0.01018746 

0.00441275 



Interaction plots for scenariorenvironment, scenario: mode, and environment:mode. 
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ANOVA for Rounds Fired Red 



Analysis of Variance Table 
Response: MOE 



Terms added sequentially (first to last) 





Df 


Sum of Sq 


Mean Sq 


F Value 


Seen 


2 


41699636 


20849818 


943.9545 


Env 


1 


660677 


660677 


29.9115 


Mode 


1 


65 


65 


0 . 0029 


Seen : Env 


2 


1278612 


639306 


28 . 9439 


Seen : Mode 


2 


180512 


90256 


4 . 0863 


Env : Mode 


1 


30401 


30401 


1.3764 


Seen : Env : Mode 


2 


367668 


183834 


8 . 3229 


Residuals 


108 


2385475 


22088 





Pr (F) 
0 .0000000 
0 . 0000003 
0.9569931 
0.0000000 
0 . 0194671 
0.2433019 
0 . 0004348 



Interaction plots for scenario:environment, scenario:mode, and environment:mode. 




Scenario # Scenario # 
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Iascam3 hi ia <ascam3 swa.ja tascam2.hl.ja fascam2.awa.ja lascaml.hl.ja 



APPENDIX F. NORMAL QUANTILE-QUANTILE PLOTS 



FASCAM.Scen #1, SWA, Janus 



FASCAM.Scen #1,SWA,JLINK 




FASCAM.Scen #1,HL, Janus 



FASCAM.Scen #1 .HL.JLINK 




-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 




-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 



Quantiles o I Standard Normal 



Quantiles of Standard Normal 



FASCAM.Scen #2, SWA, Janus 




-0.5 0.0 0.5 

Quantiles o< Standard Normal 





FASCAM.Scen #3, SWA, Janus 



FASCAM.Scen #3,SWA,JLINK 
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cham3 N j« ctiam3 awa fa eham2 hl .|a Chem2 swa ja Chaml hl ja chamlswaja 



Chem Kills, Seen #1,SWA,Janus 



Chem Kills, Seen #1, SWA, JUNK 






Chem Kills, Seen #2, HL, Janus 




•1.5 -10 -OS 0 0 OS 1.0 1.5 

Quanklaa o f Standard Norma) 



Chem Kilts, Seen #2,HL,JLINK 




•1.5 -1.0 -0.5 0.0 05 1.0 1.5 

Quanklaa of Standard Normal 




Chem Kills.Scen #3, HL, Janus 



Chem Kills.Scen #3.HL,JLINK 





•1.5 -1.0 -0.5 00 05 1 0 1.5 



Quanklaa Of Standard Normal 



Ouanklas of Standard Normal 
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Iar3 N ja Ier3.swa.je Ier2 hl.ja l^.swa)* l.fl .W.ja larlswaia 



LER.Scen #1, SWA, Janus 



LER.Scen #1,SWA,JLINK 




LER.Scen #1 ,HL, Janus LER.Scen #1 .HL.JLINK 




-1.5 -1.0 -05 0.0 0.5 1.0 1.5 -1.5 -1.0 -0.S 0.0 0.5 1.0 1.5 

Quantles ol Standard Normal Quantiles of Standard Normal 



LER.Scen #2, SWA, Janus 



LER.Scen #2,SWA,JLINK 




-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 




•1.5 -1.0 -0.5 0 0 0.5 1.0 1.5 



Quantiles of Standard Normal 



Quantiles of Standard Normal 



LER.Scen #2,HL.Janus LER.Scen #2,HL.JLINK 




-1.5 -1.0 -05 0.0 0.5 1.0 1.5 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 

Quantiles of Standard Normal Quantles of Standard Normal 



LER.Scen #3, SWA, Janus LER.Scen #3,SWA,JLINK 



— — ' — " * 


I s 

c 

• o 

o 


^ — 


-1.5 -1.0 -0.5 0 0 0.5 1 0 1.5 




-1.S -1.0 -0.5 0.0 0.5 1.0 1.5 


Quantiles of Standard Normal 




Quantiles of Standard Normal 


LER.Scen #3,HL.Janus 




LER.Scen #3,HL,JLINK 




■=. °. 

2 « 

e 

“1 


: 



-IS -1.0 -0.5 00 0.5 1.0 1.5 



-1.5 -1.0 -0.5 0.0 0.5 1 0 1.S 



Quantiles of Standard Normal 



Quantiles of Standard Normal 
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detmg3 hi b ja dettng3 swa b |e detrng2hlb|8 detmo2 swa.bja detmgVhlb.|a detmglswabia 



Det Range, Seen #1, SWA, Blue, Janus 



Det Range, Seen #1,SWA,Blue,JLINK 





1 *5 

l * 

1 o 

*o ^ 




•1.S -1.0 -0.5 0.0 05 1.0 1.5 

Quanbles of Standard Normal 

Det Range. Seen #1 ,HL,Blue,Janus 




-1.5 -1.0 -0.5 0 0 0 5 1.0 VS 

Quantiles of Standard Normal 

Det Range, Seen #1,HL,Blue,JLINK 




I 5 

0 

E 

1 s 

o 




-1.5 -1.0 -O.S 0 0 05 1.0 1.5 

Quanbles of Standard Normal 

Det Range, Seen #2,SWA,Blue.Janus 




-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 

Quanbles of Standard Normal 

Det Range.Scen #2.SWA,Blue.JL!NK 




h 

I o 

£ - 

E 

1 . 






• 



-O.S 00 OS 

Quantiles of Standard Normal 



Quantiles of Standard Normal 




Det Range, Seen #3, SWA, Blue, Janus 



Det Range, Seen #3, SWA, Blue, JUNK 




•1.5 -1 0 -05 00 O.S 1.0 1.5 

Quantiles of Standard Normal 




•IS -1.0 -O.S 00 O.S 1.0 IS 
Quantiles of Standard Normal 
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detrng3 hi r.ja detrng3.swa.r |e detmg2.hl.r.ja datrng2.swa.r ja detrngl .hl r.|a detrngl.swa r.ja 



Det Range, Seen #1,SWA,Red, Janus 



Det Range, Seen #1, SWA, Red, JUNK 




- 0.5 0.0 0.5 

Quantiles ot Standard Normal 



f T 




- 0.5 0.0 0.5 

Quantiles ot Standard Normal 




Det Range, Seen #2, SWA, Red, Janus Det Range, Seen #2,SWA,Red,JLINK 




• 1.5 - 1.0 - 0.5 0.0 0.5 1.0 1.5 - 1.5 - 1.0 - 0.5 0.0 0.5 1.0 1.5 



Quantiles ot Standard Normal 



Quantiles of Standard Normal 
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killrng3 hi b ja k(tlrng3 swa b.ja killrng2 hi b |a killmg2 swab |a kjllmgl hl.b.ja killmgl swa.b ja 



Kill Range, Seen #1,SWA,Blue t Janus Kill Range.Scen #1,SWA,Blue,JLINK 





-n o 

* £ 

“ o 

12 £ 




-1.5 -1.0 -0.5 00 05 1 0 1.5 

Quanblts of Standard Normal 

Kill Range.Scen #1,HL, Blue, Janus 


z> 

I „ 

3 CM 


-1.5 -1.0 -0 5 0 0 0 5 1.0 1.5 

Ouantlas of Standard Normal 

Kill Range.Scen #1,HL.BIue,JLINK 




^ ^ 


1.5 -1.0 -05 0.0 05 1.0 1.5 

Quanblas of Standard Normal 

Kill Range.Scen #2,SWA,Blue,Janus 




-1.5 1.0 -05 0.0 05 1.0 1.5 

Ouantlas of Standard Normal 

Kill Range.Scen #2, SWA, Blue, JUNK 




i 

£ 

1 £ 
3 . 




1.5 -1.0 -0.5 0.0 05 1 0 1.5 

Quanblas of Standard Normal 


-IS -1.0 -0.5 0.0 0 5 1 0 1.5 

Quanblas of Standard Normal 




Kill Range.Scen ttS.SWA.BIue.Janus 




•IS -1 0 -0.5 0 0 0.5 1.0 1.5 

Quantilas of Standard Norma! 



Kill Range.Scen #3,HL, Blue, Janus 



•1.5 10 -0.5 0.0 05 VO 1.5 

Ouantilos of Standard Normal 





-1.5 -VO -0.5 00 0.5 1.0 1.5 

Quanlilas of Standard Normal 



Kill Range.Scen #3, HL, Blue, JUNK 




-IS -10 -05 0.0 05 1 0 1.5 

Qua mil as of Standard Norma! 
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killmg3.hl.r.|a killmg3.swa.r.|a killmg2.ht.r.ja killmg2 swarja killmgl.ht r.ja Wltmgl swa r.|a 



Kill Range.Scen #1, SWA, Red, Janus 



Kill Range.Scen #1 .SWA.Red.JUNK 




Quanblos at Standard Normal 



Kill Range.Scen #1,HL, Red, Janus 




.1.0 -o.s o.o o.s 

Quanblas at Standard Normal 

Kill Range.Scen #2, SWA, Red, Janus 




-i.o -o.s o.o o.s 1.0 

Quanblas at Standard Normal 

Kill Range.Scen #1 .HL, Red, JUNK 




-O.S 0.0 0.5 

Quanblos at Standard Normal 

Kill Range.Scen #2, SWA, Red, JUNK 





i " 






t . 

IS CM 


— ■* • r ~~ r "" 



-1.5 -1.0 -O.S 0.0 0.5 

Quanblas at Standard Normal 



-1.0 -O.S 0.0 O.S 
Quanblas at Standard Norma! 



-1.0 -O.S 0.0 O.S 1.0 

Quanblas ol Standard Normal 

Kill Range.Scen #3,HL, Red, Janus 



-1.0 -0.5 0.0 O.S 1.0 

Ouanblas at Standard Normal 

Kill Range.Scen #3, HL, Red, JUNK 




-1.0 -O.S 0.0 o.s 

Quanblas ot Standard Normal 



Kill Range.Scen #2, HL, Red, Janus 




Kill Range.Scen #2,HL,Red,JLINK 




^ o 

' 1 5 
| 






15 8 




-1.5 -1.0 -O.S 0.0 0.5 1.0 1.5 

Quanblas ot Standard Normal 

Kill Range.Scen #3, SWA, Red, Janus 




<l7 -1.0 -0.5 0.0 0.5 1.0 1.5 

Quanblas ot Standard Normal 

Kill Range.Scen #3, SWA, Red, JUNK 




I - 
5 


— 




kilim 

1.30 






Quanblas ot Standard Normal 
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rnd&3 hi b |s rnd»3 swa b ja rnds2hlbja mds2.swab|a rndsl.ht.b.ja rndst.swabja 



Rnds Fired, Seen #1 ,SWA,Blue,Janus 



Rnds Fired.Scen #1,SWA,Blue,JLINK 





- 1.5 - 1.0 - 0.5 0.0 0.5 1.0 1.5 



Quantiles of Standard Normal 



Quantiles of Standard Normal 



Rnds Fired.Scen #1,HL, Blue, Janus 




Quantllas of Standard Normal 



Rnds Fired.Scen #1 ,HL, Blue, JUNK 




- 1.5 - 1.0 - 0.5 0.0 0.5 1.0 1.5 

Quantiles of Standard Normal 



Rnds Fired.Scen #2,SWA,Blue,Janus 




- 1.5 - 1.0 - 0.5 0.0 0.5 1.0 1 5 



Rnds Fired.Scen #2,SWA,Blue,JLINK 




- 1.5 - 1.0 -05 0.0 0.5 1.0 1.5 



Quantllas of Standard Normal 



Quantiles of Standard Normal 




Rnds Fired.Scen #3. SWA, Blue, Janus 




- 1.5 -1 0 - 0.5 0 0 0 5 1.0 1 5 



Rnds Fired.Scen #3, SWA, Blue, JUNK 




• 1.5 - 1.0 - 0.5 0.0 0.5 1.0 1.5 



Quantiles of Standard Normal 



Quantiles ol Standard Normal 
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APPENDIX G. KOLMOGOROV-SMIRNOV GOODNESS OF FIT RESULTS 



Highlighted portions are those MOPs which produce p-values<0.05 (a=0.05), and 
thus reject the null hypothesis that the sample comes from a Normal distribution. 



Scenario#!, SWA 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JUNK 


Janus 


JUNK 


Janas 


JUNK 


P-Value 


.60 


.75 


.04 


.03 


.44 


.73 


Scenario #1,HL 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


P-Value 


.92 


.63 


.47 


.42 


.43 


.84 


Scenario #2, SWA 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


P-Value 


.48 


.84 


.20 


.004 


.79 


.89 


Scenario #2, HL 




FASCAM 


Chem Kills 


LER 


Mode 


Janas 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


P-Value 


.14 


.08 


.91 


.44 


.74 


.99 


Scenario #3, SWA 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JUNK 


Janas 


JUNK 


Janus 


JUNK 


P-Value 


.46 


m 


.51 


.01 


.59 


.95 


Scenario #3, HL 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JUNK 


Janas 


JUNK 


Janas 


JUNK 


P-Value 


.60 


.08 


,001 


.93 


.38 


.93 



ill 



Scenario #1,SWA 




Dot Rng Blue 


Del Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Fired Blue 


Rnds Hied Red 


Mode 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


P- Value 


.98 


.93 


51 


.25 


.10 


.84 


.94 


.44 


.63 


.24 


.93 


.89 


Scenario#!, HI, 




Del Rng Blue 


Del Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Hied Blue 


Rnds Hied Red 


Mode 


Janus 


JUNK 


Janus 


JUNK 


Janas 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


P-Value 


.97 


.65 


.56 


.92 


.99 


.95 


.99 


.75 


.19 


.45 


.94 


.98 


Scenario #2, SWA 




Del Rng Blue 


Del Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Hied Blue 


Rnds Hied Red 


Mode 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


P-Value 


.41 


.21 


.78 


.95 


.08 


.10 


.92 


.91 


.97 


.19 


.40 


.85 


Scenario #2, HL 




Del Rng Blue 


Dei Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Hied Blue 


Rnds Fired Red 


Mode 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


P-Value 


.78 


.58 


.96 


.96 


.46 


.85 


.82 


.43 


.80 


.58 


.97 


.81 


Scenario #3, SWA 




Del Rng Blue 


Del Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Hied Blue 


Rnds Fired Red 


Mode 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


P-Value 


.68 


.52 


.92 


.64 


.98 


26 


.98 


.76 


.50 


.92 


.82 


.90 


Scenario #3, HL 




Del Rng Blue 


Del Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Hied Blue 


Rnds Fired Red 


Mode 


Janus 


JUNK 


Janus 


JUNK 


Janas 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


P-Value 


.47 


.97 


.33 


.87 


.70 


.44 


.97 


.98 


.29 


.94 


.97 


.91 
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APPENDIX H. F-TEST FOR COMMON VARIANCE RESULTS 



Highlighted portions are those MOPs which produce p-values<0.05 (a=0.05), and 
thus reject the null hypothesis that the two samples share a common variance. 



Scenario #1, SWA 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


P-Value 


.31 


.29 


.97 


Scenario #1, HL 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


P-Value 


.045 


.24 


.10 


Scenario #2, SWA 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


P-Value 


.46 


.051 


.14 


Scenario #2, HL 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


P-Value 


.003 


.15 


.11 


Scenario #3, SWA 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


P-Value 


.01 


.01 


.00 


Scenario #3> HL 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


P-Value 


.045 


.00 


.23 
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Scenario#!, SWA 




DetRngBlue 


Det Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Fired Blue 


Rnds Fired Red 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


P-Value 


.39 


.73 


.00 


37 


.00 


.76 


Scenario #1,HL 




Det Rng Blue 


Det Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Fired Blue 


Rnds Fired Red 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


P-Value 


.24 


.13 


.66 


.23 


.00 


.00 


Scenario #2, SWA 




Det Rng Blue 


Det Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Fired Blue 


Rnds Fired Red 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


P-Value 


.12 


.19 


50 


.17 


.82 


.01 


Scenario #2, HL 




DetRngBlue 


Det Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Fired Blue 


Rnds Fired Red 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


P-Value 


.45 


.00 


.21 


31 


.94 


.99 


Scenario #3, SWA 




Det Rng Blue 


Det Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Fired Blue 


Rnds Fired Red 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


P-Value 


.25 


.98 


.01 


.98 


,00 


.17 


Scenario #3, HL 




Det Rng Blue 


Det Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Fired Blue 


Rnds Fired Red 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


P-Value 


.84 


.22 


.02 


.05 


.12 


.78 
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APPENDIX I. /-TEST RESULTS 



Highlighted portions are those MOPs which produce p-values<0.0083, 

(oc new = a old / 6 = 0.0083) and thus reject the null hypothesis that the two sample means are 
statistically similar. Also, the 1 -OC new =99.17 percent confidence interval (Cl) is provided to 
show the extent to which the sample means are similar (dissimilar). All MOPs failing to 
reject the null hypothesis (p-value>0.0083) produce CIs which include zero. 
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Scenario #1,SWA 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Cl 


(-1.18,258) 




(.66,1.38) 


p-value 


.28 


na 


.00 


Scenario #1, HL 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


ci 




(7.81,11.79) 


(-.27,58) 


p- value 


na 


,00 


30 


Scenario #2, SWA 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Cl 


(-1.14,2.74) 




(-.09,. 49) 


p-value 


.24 


na 


.05 


Scenario #2, HL 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Cl 






(-.72,. 03) 


p-value 


na 


na 


.014 


Scenario #3, SWA 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


p-value 


na 


na 


na 


Scenario #3, HL 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Cl 






(-.37.1.88) 


p-value 


na 


na 


.06 
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Scenario#!, SWA 




Del Rng Blue 


Det Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Fired Blue 


Rnds Fired Red 


Mode 


Janas 


JLink 


Janas 


JLink 


Janas 


JLink 


Janas 


JLink 


Janas 


JLink 


Janus 


JLink 


Cl 


(-.60,-37) 


(-1.1 3, -.71) 


(-.66,. 02) 


(-.54,. 03) 


(17.52,87.48) 


(-129.14,41.26) 


P-Valut 


.00 


.00 


.02 


.02 


.00 


.00 


Scenario#!, HL 




Del Rng Blue 


Det Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Fired Blue 


Rnds Fired Red 


Mode 


Janus 


JLink 


Janas 


JLink 


Janus 


JLink 


Janas 


JLink 


Janus 


JLink 


Janas 


JLink 


Cl 


(-.74, -.50) 


(-1.08, -.68) 


(-.91,-30) 


(-.53,, 25) 


(19.14,89.26) 


(-60.62 , 476.82) 


P-Valuf 


.00 


,00 


.00 


.00 


.00 


.03 


Scenario #2, SWA 




D et Rng Blue 


Det Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Fired Blue 


Rnds Fired Red 


Mode 


Janas 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Cl 


(-.65, -.26) 


(-1.21, -.85) 


(-.90,. 22) 


(-.28,. 28) 


(2.88,39.52) 


(-104.78,61.38) 


P-Valuc 


.00 


.00 


.09 


.99 


.003 


.45 


Scenario #2, HL 




Del Rng Blue 


Det Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Fired Blue 


Rnds Fired Red 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Cl 


(-.70, -.47) 


(-.97,, 51) 


(-.70,. 07) 


(-54 , -.23) 


(-28.42,24.42) 


(-71.46,294.46) 


P- Value 


.00 


.00 


.03 


.00 


.83 


.09 


Scenario #3, SWA 




Det Rng Blue 


Det Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Fired Blue 


Rnds Fired Red 


Mode 


Janas 


JLink 


Janus 


JLink 


Janus 


JLink 


Janas 


JLink 


Janas 


JLink 


Janus 


JUnk 


Cl 


(-.05,. 04) 


(-.26,-. 18) 


(-.22,, 08) 


(-.14,. 03) 


(109.48,587.12) 


(-111.99,125.99) 


P-Value 


.63 


.00 


.00 


.07 


,00 


.86 


Scenario #3, HL 




Det Rng Blue 


Det Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Fired Blue 


Rnds Fired Red 


Mode 


Janas 


JLink 


Janas 


JLink 


Janas 


JLink 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


Cl 


(-.12,-03) 


(-.16,, 07) 


(-.13,, 02) 


(-.28,. 001) 


(-324.35,336.55) 


(-550.96,93.96) 


P-Valut 


.00 


.00 


.001 


.009 


.96 


.05 
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APPENDIX J. WILCOXEN TEST RESULTS 



Highlighted portions those MOPs which produce p-values<0.0083, ( 0t olJ / 6 = 0.0083 ) 
and thus reject the null hypothesis that the two sample means are statistically similar. 



Scenario#!, SWA 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JLink 


Janus 


JLink 


Janas 


JLink 


p- value 


.29 


.000 


.000 


Scenario #1,HL 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JLink 


Janus 


JLink 


Janas 


JLink 


p-value 


m 


.000 


.47 


Scenario #2, SWA 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


p-value 


.26 


.000 


.05 


Scenario #2, HL 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


p-value 


.001 


.000 


.02 


Scenario #3, SWA 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


p-value 


.08 


.000 


.001 


Scenario #3, HL 




FASCAM 


Chem Kills 


LER 


Mode 


Janus 


JLink 


Janus 


JLink 


Janus 


JLink 


p-value 


.49 


.000 


.04 
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Scenario #1, SWA 




Det Rng Blue 


Det Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Filed Blue 


Rnds Fired Red 


Mode 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


p- value 


.000 


.000 


.0015 


.02 


.000 


.000 


Scenario #1, HL 




Det Rng Blue 


Det Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Fired Blue 


Rnds Fired Red 


Mode 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


p-value 


.000 


.000 


.000 


.000 


.000 


.08 


Scenario #2, SWA 




Det Rng Blue 


Det Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Fired Blue 


Rnds Fired Red 


Mode 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


p-value 


.000 


.000 


.01 


.74 


.005 


12 


Scenario #2, HL 




Det Rng Blue 


Det Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Fired Blue 


Rnds Fired Red 


Mode 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


p-value 


.000 


.000 


.04 


.000 


.97 


.08 


Scenario #3, SWA 




Det Rng Blue 


Det Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Fired Blue 


Rnds Fired Red 


Mode 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


p-value 


.48 


.000 


.000 


.11 


.000 


.94 


Scenario #3, HL 




Det Rng Blue 


Det Rng Red 


Kill Rng Blue 


Kill Rng Red 


Rnds Fired Blue 


Rnds Fired Red 


Mode 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


Janus 


JUNK 


p-value 


.000 


.000 


.0011 


.009 


.48 


.06 
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